2023-02-10 07:53:59,873 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-10 07:54:00,126 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d 2023-02-10 07:54:00,143 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-10 07:54:00,143 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063, deleteOnExit=true 2023-02-10 07:54:00,144 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-10 07:54:00,144 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/test.cache.data in system properties and HBase conf 2023-02-10 07:54:00,145 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/hadoop.tmp.dir in system properties and HBase conf 2023-02-10 07:54:00,145 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/hadoop.log.dir in system properties and HBase conf 2023-02-10 07:54:00,146 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-10 07:54:00,147 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-10 07:54:00,147 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-10 07:54:00,273 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-10 07:54:00,671 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-10 07:54:00,676 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-10 07:54:00,677 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-10 07:54:00,677 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-10 07:54:00,678 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-10 07:54:00,678 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-10 07:54:00,679 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-10 07:54:00,679 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-10 07:54:00,680 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-10 07:54:00,680 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-10 07:54:00,681 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/nfs.dump.dir in system properties and HBase conf 2023-02-10 07:54:00,681 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/java.io.tmpdir in system properties and HBase conf 2023-02-10 07:54:00,681 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-10 07:54:00,682 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-10 07:54:00,682 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-10 07:54:01,181 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-10 07:54:01,185 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-10 07:54:01,479 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-10 07:54:01,663 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-10 07:54:01,683 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 07:54:01,724 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 07:54:01,756 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/java.io.tmpdir/Jetty_localhost_45059_hdfs____.gyfyxv/webapp 2023-02-10 07:54:01,930 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45059 2023-02-10 07:54:01,942 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-10 07:54:01,942 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-10 07:54:02,333 WARN [Listener at localhost/34635] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 07:54:02,433 WARN [Listener at localhost/34635] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 07:54:02,453 WARN [Listener at localhost/34635] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 07:54:02,459 INFO [Listener at localhost/34635] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 07:54:02,464 INFO [Listener at localhost/34635] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/java.io.tmpdir/Jetty_localhost_43893_datanode____qfh2ai/webapp 2023-02-10 07:54:02,564 INFO [Listener at localhost/34635] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43893 2023-02-10 07:54:02,881 WARN [Listener at localhost/41913] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 07:54:02,893 WARN [Listener at localhost/41913] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 07:54:02,895 WARN [Listener at localhost/41913] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 07:54:02,897 INFO [Listener at localhost/41913] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 07:54:02,905 INFO [Listener at localhost/41913] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/java.io.tmpdir/Jetty_localhost_44903_datanode____.nw6m59/webapp 2023-02-10 07:54:03,007 INFO [Listener at localhost/41913] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44903 2023-02-10 07:54:03,016 WARN [Listener at localhost/38035] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 07:54:03,032 WARN [Listener at localhost/38035] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-10 07:54:03,036 WARN [Listener at localhost/38035] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-10 07:54:03,037 INFO [Listener at localhost/38035] log.Slf4jLog(67): jetty-6.1.26 2023-02-10 07:54:03,043 INFO [Listener at localhost/38035] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/java.io.tmpdir/Jetty_localhost_34529_datanode____l93jc2/webapp 2023-02-10 07:54:03,184 INFO [Listener at localhost/38035] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34529 2023-02-10 07:54:03,200 WARN [Listener at localhost/43795] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-10 07:54:03,413 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa67fee913fb96b65: Processing first storage report for DS-919a5c51-ddc0-4069-8c7d-571f4c11934e from datanode 929ae40a-10a1-4838-a0c1-b6853964e24b 2023-02-10 07:54:03,415 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa67fee913fb96b65: from storage DS-919a5c51-ddc0-4069-8c7d-571f4c11934e node DatanodeRegistration(127.0.0.1:38371, datanodeUuid=929ae40a-10a1-4838-a0c1-b6853964e24b, infoPort=35127, infoSecurePort=0, ipcPort=41913, storageInfo=lv=-57;cid=testClusterID;nsid=586786342;c=1676015641262), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-10 07:54:03,415 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd994569c88acda47: Processing first storage report for DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4 from datanode 1d48e5e3-95b6-462f-8431-b0ebec0b2069 2023-02-10 07:54:03,415 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd994569c88acda47: from storage DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4 node DatanodeRegistration(127.0.0.1:34997, datanodeUuid=1d48e5e3-95b6-462f-8431-b0ebec0b2069, infoPort=41223, infoSecurePort=0, ipcPort=38035, storageInfo=lv=-57;cid=testClusterID;nsid=586786342;c=1676015641262), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 07:54:03,416 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd039cd62f9cc169b: Processing first storage report for DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0 from datanode e71256cb-90c7-4766-bfdc-18ad52545fac 2023-02-10 07:54:03,416 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd039cd62f9cc169b: from storage DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0 node DatanodeRegistration(127.0.0.1:43743, datanodeUuid=e71256cb-90c7-4766-bfdc-18ad52545fac, infoPort=42593, infoSecurePort=0, ipcPort=43795, storageInfo=lv=-57;cid=testClusterID;nsid=586786342;c=1676015641262), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-10 07:54:03,416 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa67fee913fb96b65: Processing first storage report for DS-b1aa24c6-f414-4549-b89a-3eb7fd89216a from datanode 929ae40a-10a1-4838-a0c1-b6853964e24b 2023-02-10 07:54:03,416 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa67fee913fb96b65: from storage DS-b1aa24c6-f414-4549-b89a-3eb7fd89216a node DatanodeRegistration(127.0.0.1:38371, datanodeUuid=929ae40a-10a1-4838-a0c1-b6853964e24b, infoPort=35127, infoSecurePort=0, ipcPort=41913, storageInfo=lv=-57;cid=testClusterID;nsid=586786342;c=1676015641262), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 07:54:03,416 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd994569c88acda47: Processing first storage report for DS-6ca84026-d9e2-467d-818c-80a3ac3de73c from datanode 1d48e5e3-95b6-462f-8431-b0ebec0b2069 2023-02-10 07:54:03,416 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd994569c88acda47: from storage DS-6ca84026-d9e2-467d-818c-80a3ac3de73c node DatanodeRegistration(127.0.0.1:34997, datanodeUuid=1d48e5e3-95b6-462f-8431-b0ebec0b2069, infoPort=41223, infoSecurePort=0, ipcPort=38035, storageInfo=lv=-57;cid=testClusterID;nsid=586786342;c=1676015641262), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 07:54:03,417 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd039cd62f9cc169b: Processing first storage report for DS-16b0af81-dd7c-4bf1-bb59-d4f323518ce1 from datanode e71256cb-90c7-4766-bfdc-18ad52545fac 2023-02-10 07:54:03,417 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd039cd62f9cc169b: from storage DS-16b0af81-dd7c-4bf1-bb59-d4f323518ce1 node DatanodeRegistration(127.0.0.1:43743, datanodeUuid=e71256cb-90c7-4766-bfdc-18ad52545fac, infoPort=42593, infoSecurePort=0, ipcPort=43795, storageInfo=lv=-57;cid=testClusterID;nsid=586786342;c=1676015641262), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-10 07:54:03,609 DEBUG [Listener at localhost/43795] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d 2023-02-10 07:54:03,671 INFO [Listener at localhost/43795] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/zookeeper_0, clientPort=60412, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-10 07:54:03,688 INFO [Listener at localhost/43795] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=60412 2023-02-10 07:54:03,699 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:03,701 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:04,366 INFO [Listener at localhost/43795] util.FSUtils(479): Created version file at hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1 with version=8 2023-02-10 07:54:04,367 INFO [Listener at localhost/43795] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/hbase-staging 2023-02-10 07:54:04,682 INFO [Listener at localhost/43795] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-10 07:54:05,127 INFO [Listener at localhost/43795] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-10 07:54:05,158 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,159 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,159 INFO [Listener at localhost/43795] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 07:54:05,160 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,160 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 07:54:05,301 INFO [Listener at localhost/43795] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 07:54:05,381 DEBUG [Listener at localhost/43795] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-10 07:54:05,481 INFO [Listener at localhost/43795] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37109 2023-02-10 07:54:05,491 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,493 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,513 INFO [Listener at localhost/43795] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37109 connecting to ZooKeeper ensemble=127.0.0.1:60412 2023-02-10 07:54:05,552 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:371090x0, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 07:54:05,555 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37109-0x1003d3915cd0000 connected 2023-02-10 07:54:05,578 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 07:54:05,579 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:54:05,582 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 07:54:05,589 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37109 2023-02-10 07:54:05,590 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37109 2023-02-10 07:54:05,590 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37109 2023-02-10 07:54:05,591 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37109 2023-02-10 07:54:05,591 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37109 2023-02-10 07:54:05,597 INFO [Listener at localhost/43795] master.HMaster(439): hbase.rootdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1, hbase.cluster.distributed=false 2023-02-10 07:54:05,663 INFO [Listener at localhost/43795] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-10 07:54:05,663 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,663 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,663 INFO [Listener at localhost/43795] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 07:54:05,663 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,663 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 07:54:05,668 INFO [Listener at localhost/43795] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 07:54:05,671 INFO [Listener at localhost/43795] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43281 2023-02-10 07:54:05,673 INFO [Listener at localhost/43795] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 07:54:05,679 DEBUG [Listener at localhost/43795] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 07:54:05,681 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,682 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,684 INFO [Listener at localhost/43795] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43281 connecting to ZooKeeper ensemble=127.0.0.1:60412 2023-02-10 07:54:05,688 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:432810x0, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 07:54:05,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43281-0x1003d3915cd0001 connected 2023-02-10 07:54:05,689 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 07:54:05,691 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:54:05,692 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 07:54:05,692 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43281 2023-02-10 07:54:05,693 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43281 2023-02-10 07:54:05,693 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43281 2023-02-10 07:54:05,693 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43281 2023-02-10 07:54:05,693 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43281 2023-02-10 07:54:05,706 INFO [Listener at localhost/43795] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-10 07:54:05,706 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,707 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,707 INFO [Listener at localhost/43795] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 07:54:05,707 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,707 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 07:54:05,708 INFO [Listener at localhost/43795] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 07:54:05,709 INFO [Listener at localhost/43795] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43211 2023-02-10 07:54:05,710 INFO [Listener at localhost/43795] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 07:54:05,711 DEBUG [Listener at localhost/43795] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 07:54:05,711 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,713 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,715 INFO [Listener at localhost/43795] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43211 connecting to ZooKeeper ensemble=127.0.0.1:60412 2023-02-10 07:54:05,719 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:432110x0, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 07:54:05,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43211-0x1003d3915cd0002 connected 2023-02-10 07:54:05,720 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 07:54:05,721 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:54:05,722 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 07:54:05,722 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43211 2023-02-10 07:54:05,722 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43211 2023-02-10 07:54:05,723 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43211 2023-02-10 07:54:05,724 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43211 2023-02-10 07:54:05,724 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43211 2023-02-10 07:54:05,737 INFO [Listener at localhost/43795] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-10 07:54:05,737 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,737 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,737 INFO [Listener at localhost/43795] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-10 07:54:05,737 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-10 07:54:05,737 INFO [Listener at localhost/43795] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-10 07:54:05,737 INFO [Listener at localhost/43795] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-10 07:54:05,739 INFO [Listener at localhost/43795] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40867 2023-02-10 07:54:05,739 INFO [Listener at localhost/43795] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-10 07:54:05,740 DEBUG [Listener at localhost/43795] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-10 07:54:05,741 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,743 INFO [Listener at localhost/43795] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,744 INFO [Listener at localhost/43795] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:40867 connecting to ZooKeeper ensemble=127.0.0.1:60412 2023-02-10 07:54:05,748 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:408670x0, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-10 07:54:05,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:40867-0x1003d3915cd0003 connected 2023-02-10 07:54:05,749 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 07:54:05,750 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:54:05,751 DEBUG [Listener at localhost/43795] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-10 07:54:05,751 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40867 2023-02-10 07:54:05,751 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40867 2023-02-10 07:54:05,752 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40867 2023-02-10 07:54:05,752 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40867 2023-02-10 07:54:05,752 DEBUG [Listener at localhost/43795] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40867 2023-02-10 07:54:05,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:05,763 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-10 07:54:05,765 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:05,782 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 07:54:05,782 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 07:54:05,782 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 07:54:05,782 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-10 07:54:05,783 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:05,783 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-10 07:54:05,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37109,1676015644499 from backup master directory 2023-02-10 07:54:05,785 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-10 07:54:05,789 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:05,789 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-10 07:54:05,789 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 07:54:05,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:05,792 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-10 07:54:05,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-10 07:54:05,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/hbase.id with ID: aa37d2b9-16a1-4502-8a65-61d82311e464 2023-02-10 07:54:05,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-10 07:54:05,946 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:05,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5baf9fd9 to 127.0.0.1:60412 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 07:54:06,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@dd36587, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 07:54:06,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-10 07:54:06,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-10 07:54:06,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-10 07:54:06,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-10 07:54:06,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-10 07:54:06,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-10 07:54:06,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 07:54:06,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store-tmp 2023-02-10 07:54:06,185 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:06,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-10 07:54:06,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:54:06,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:54:06,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-10 07:54:06,186 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:54:06,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:54:06,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676015646186Disabling compacts and flushes for region at 1676015646186Disabling writes for close at 1676015646186Writing region close event to WAL at 1676015646186Closed at 1676015646186 2023-02-10 07:54:06,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/WALs/jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:06,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37109%2C1676015644499, suffix=, logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/WALs/jenkins-hbase4.apache.org,37109,1676015644499, archiveDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/oldWALs, maxLogs=10 2023-02-10 07:54:06,269 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK] 2023-02-10 07:54:06,269 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK] 2023-02-10 07:54:06,269 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK] 2023-02-10 07:54:06,277 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-10 07:54:06,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/WALs/jenkins-hbase4.apache.org,37109,1676015644499/jenkins-hbase4.apache.org%2C37109%2C1676015644499.1676015646223 2023-02-10 07:54:06,348 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK], DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK]] 2023-02-10 07:54:06,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:06,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:06,352 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 07:54:06,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 07:54:06,412 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-10 07:54:06,418 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-10 07:54:06,442 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-10 07:54:06,454 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:06,459 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-10 07:54:06,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-10 07:54:06,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-10 07:54:06,480 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:06,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75424588, jitterRate=0.12391394376754761}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 07:54:06,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676015646380Initializing all the Stores at 1676015646382 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015646383 (+1 ms)Cleaning up temporary data from old regions at 1676015646467 (+84 ms)Cleaning up detritus from prior splits at 1676015646468 (+1 ms)Region opened successfully at 1676015646481 (+13 ms) 2023-02-10 07:54:06,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-10 07:54:06,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:06,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:06,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:06,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-10 07:54:06,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-02-10 07:54:06,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-10 07:54:06,563 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-10 07:54:06,568 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-10 07:54:06,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-10 07:54:06,603 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-10 07:54:06,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-10 07:54:06,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-10 07:54:06,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-10 07:54:06,618 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:06,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-10 07:54:06,621 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-10 07:54:06,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-10 07:54:06,637 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 07:54:06,637 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 07:54:06,637 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 07:54:06,637 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-10 07:54:06,638 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:06,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,37109,1676015644499, sessionid=0x1003d3915cd0000, setting cluster-up flag (Was=false) 2023-02-10 07:54:06,658 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:06,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-10 07:54:06,666 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:06,670 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:06,676 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-10 07:54:06,677 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:06,680 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.hbase-snapshot/.tmp 2023-02-10 07:54:06,756 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(952): ClusterId : aa37d2b9-16a1-4502-8a65-61d82311e464 2023-02-10 07:54:06,756 INFO [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(952): ClusterId : aa37d2b9-16a1-4502-8a65-61d82311e464 2023-02-10 07:54:06,756 INFO [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(952): ClusterId : aa37d2b9-16a1-4502-8a65-61d82311e464 2023-02-10 07:54:06,763 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 07:54:06,763 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 07:54:06,763 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-10 07:54:06,770 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 07:54:06,770 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 07:54:06,770 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-10 07:54:06,770 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 07:54:06,770 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 07:54:06,770 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-10 07:54:06,776 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 07:54:06,776 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 07:54:06,777 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-10 07:54:06,780 DEBUG [RS:0;jenkins-hbase4:43281] zookeeper.ReadOnlyZKClient(139): Connect 0x2b18cf39 to 127.0.0.1:60412 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 07:54:06,780 DEBUG [RS:2;jenkins-hbase4:40867] zookeeper.ReadOnlyZKClient(139): Connect 0x231c9e78 to 127.0.0.1:60412 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 07:54:06,780 DEBUG [RS:1;jenkins-hbase4:43211] zookeeper.ReadOnlyZKClient(139): Connect 0x7bc645b4 to 127.0.0.1:60412 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 07:54:06,792 DEBUG [RS:0;jenkins-hbase4:43281] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@25df342a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 07:54:06,793 DEBUG [RS:2;jenkins-hbase4:40867] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ddb646b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 07:54:06,793 DEBUG [RS:1;jenkins-hbase4:43211] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7501b4bc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 07:54:06,793 DEBUG [RS:2;jenkins-hbase4:40867] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29e4f508, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 07:54:06,793 DEBUG [RS:0;jenkins-hbase4:43281] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21e0a449, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 07:54:06,793 DEBUG [RS:1;jenkins-hbase4:43211] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@713e2ec4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 07:54:06,800 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:06,810 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 07:54:06,810 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 07:54:06,810 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 07:54:06,810 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-10 07:54:06,810 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-10 07:54:06,811 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:06,811 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 07:54:06,811 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:06,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015676811 2023-02-10 07:54:06,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-10 07:54:06,820 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:06,821 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-10 07:54:06,824 DEBUG [RS:0;jenkins-hbase4:43281] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43281 2023-02-10 07:54:06,825 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-10 07:54:06,825 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:40867 2023-02-10 07:54:06,825 DEBUG [RS:1;jenkins-hbase4:43211] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:43211 2023-02-10 07:54:06,827 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-10 07:54:06,830 INFO [RS:2;jenkins-hbase4:40867] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 07:54:06,830 INFO [RS:0;jenkins-hbase4:43281] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 07:54:06,831 INFO [RS:0;jenkins-hbase4:43281] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 07:54:06,831 INFO [RS:2;jenkins-hbase4:40867] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 07:54:06,830 INFO [RS:1;jenkins-hbase4:43211] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-10 07:54:06,831 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 07:54:06,831 DEBUG [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 07:54:06,832 INFO [RS:1;jenkins-hbase4:43211] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-10 07:54:06,832 DEBUG [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1023): About to register with Master. 2023-02-10 07:54:06,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-10 07:54:06,835 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37109,1676015644499 with isa=jenkins-hbase4.apache.org/172.31.14.131:40867, startcode=1676015645736 2023-02-10 07:54:06,835 INFO [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37109,1676015644499 with isa=jenkins-hbase4.apache.org/172.31.14.131:43211, startcode=1676015645706 2023-02-10 07:54:06,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-10 07:54:06,835 INFO [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37109,1676015644499 with isa=jenkins-hbase4.apache.org/172.31.14.131:43281, startcode=1676015645662 2023-02-10 07:54:06,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-10 07:54:06,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-10 07:54:06,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:06,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-10 07:54:06,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-10 07:54:06,840 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-10 07:54:06,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-10 07:54:06,843 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-10 07:54:06,844 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676015646844,5,FailOnTimeoutGroup] 2023-02-10 07:54:06,845 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676015646845,5,FailOnTimeoutGroup] 2023-02-10 07:54:06,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:06,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-10 07:54:06,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:06,847 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:06,856 DEBUG [RS:1;jenkins-hbase4:43211] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 07:54:06,856 DEBUG [RS:2;jenkins-hbase4:40867] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 07:54:06,856 DEBUG [RS:0;jenkins-hbase4:43281] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-10 07:54:06,882 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:06,883 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:06,884 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1 2023-02-10 07:54:06,923 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:06,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-10 07:54:06,928 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/info 2023-02-10 07:54:06,929 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-10 07:54:06,930 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:06,930 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-10 07:54:06,931 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59407, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 07:54:06,931 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44615, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 07:54:06,931 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49549, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-10 07:54:06,933 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/rep_barrier 2023-02-10 07:54:06,934 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-10 07:54:06,935 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:06,935 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-10 07:54:06,937 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/table 2023-02-10 07:54:06,938 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-10 07:54:06,939 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:06,941 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740 2023-02-10 07:54:06,942 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740 2023-02-10 07:54:06,943 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37109] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:06,943 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:06,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37109] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:06,947 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-10 07:54:06,949 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-10 07:54:06,953 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:06,954 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71479190, jitterRate=0.06512293219566345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-10 07:54:06,954 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676015646923Initializing all the Stores at 1676015646925 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676015646925Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015646925Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676015646925Cleaning up temporary data from old regions at 1676015646945 (+20 ms)Cleaning up detritus from prior splits at 1676015646945Region opened successfully at 1676015646954 (+9 ms) 2023-02-10 07:54:06,955 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-10 07:54:06,955 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-10 07:54:06,955 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-10 07:54:06,955 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-10 07:54:06,955 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-10 07:54:06,956 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-10 07:54:06,957 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676015646955Disabling compacts and flushes for region at 1676015646955Disabling writes for close at 1676015646955Writing region close event to WAL at 1676015646956 (+1 ms)Closed at 1676015646956 2023-02-10 07:54:06,962 DEBUG [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1 2023-02-10 07:54:06,962 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1 2023-02-10 07:54:06,962 DEBUG [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:34635 2023-02-10 07:54:06,963 DEBUG [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 07:54:06,962 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:34635 2023-02-10 07:54:06,964 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 07:54:06,963 DEBUG [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1 2023-02-10 07:54:06,964 DEBUG [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:34635 2023-02-10 07:54:06,964 DEBUG [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-10 07:54:06,966 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:06,966 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-10 07:54:06,970 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 07:54:06,973 DEBUG [RS:1;jenkins-hbase4:43211] zookeeper.ZKUtil(162): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:06,973 WARN [RS:1;jenkins-hbase4:43211] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 07:54:06,973 INFO [RS:1;jenkins-hbase4:43211] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 07:54:06,973 DEBUG [RS:2;jenkins-hbase4:40867] zookeeper.ZKUtil(162): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:06,973 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,40867,1676015645736] 2023-02-10 07:54:06,973 DEBUG [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1947): logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:06,973 DEBUG [RS:0;jenkins-hbase4:43281] zookeeper.ZKUtil(162): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:06,974 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43211,1676015645706] 2023-02-10 07:54:06,973 WARN [RS:2;jenkins-hbase4:40867] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 07:54:06,974 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43281,1676015645662] 2023-02-10 07:54:06,974 WARN [RS:0;jenkins-hbase4:43281] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-10 07:54:06,974 INFO [RS:2;jenkins-hbase4:40867] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 07:54:06,975 INFO [RS:0;jenkins-hbase4:43281] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 07:54:06,975 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1947): logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:06,975 DEBUG [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1947): logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:06,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-10 07:54:06,995 DEBUG [RS:2;jenkins-hbase4:40867] zookeeper.ZKUtil(162): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:06,995 DEBUG [RS:1;jenkins-hbase4:43211] zookeeper.ZKUtil(162): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:06,995 DEBUG [RS:0;jenkins-hbase4:43281] zookeeper.ZKUtil(162): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:06,996 DEBUG [RS:2;jenkins-hbase4:40867] zookeeper.ZKUtil(162): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:06,996 DEBUG [RS:1;jenkins-hbase4:43211] zookeeper.ZKUtil(162): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:06,996 DEBUG [RS:0;jenkins-hbase4:43281] zookeeper.ZKUtil(162): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:06,996 DEBUG [RS:2;jenkins-hbase4:40867] zookeeper.ZKUtil(162): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:06,996 DEBUG [RS:1;jenkins-hbase4:43211] zookeeper.ZKUtil(162): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:06,996 DEBUG [RS:0;jenkins-hbase4:43281] zookeeper.ZKUtil(162): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:07,004 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-10 07:54:07,006 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-10 07:54:07,007 DEBUG [RS:1;jenkins-hbase4:43211] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 07:54:07,007 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 07:54:07,007 DEBUG [RS:0;jenkins-hbase4:43281] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-10 07:54:07,017 INFO [RS:1;jenkins-hbase4:43211] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 07:54:07,018 INFO [RS:0;jenkins-hbase4:43281] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 07:54:07,017 INFO [RS:2;jenkins-hbase4:40867] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-10 07:54:07,041 INFO [RS:1;jenkins-hbase4:43211] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 07:54:07,041 INFO [RS:2;jenkins-hbase4:40867] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 07:54:07,041 INFO [RS:0;jenkins-hbase4:43281] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-10 07:54:07,045 INFO [RS:1;jenkins-hbase4:43211] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 07:54:07,045 INFO [RS:0;jenkins-hbase4:43281] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 07:54:07,046 INFO [RS:1;jenkins-hbase4:43211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,046 INFO [RS:0;jenkins-hbase4:43281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,045 INFO [RS:2;jenkins-hbase4:40867] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-10 07:54:07,046 INFO [RS:2;jenkins-hbase4:40867] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,047 INFO [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 07:54:07,047 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 07:54:07,047 INFO [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-10 07:54:07,054 INFO [RS:2;jenkins-hbase4:40867] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,054 INFO [RS:0;jenkins-hbase4:43281] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,054 INFO [RS:1;jenkins-hbase4:43211] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,055 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 07:54:07,056 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 07:54:07,056 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,055 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:1;jenkins-hbase4:43211] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-10 07:54:07,056 DEBUG [RS:2;jenkins-hbase4:40867] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,056 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,057 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,057 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,057 DEBUG [RS:0;jenkins-hbase4:43281] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-10 07:54:07,060 INFO [RS:2;jenkins-hbase4:40867] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,060 INFO [RS:1;jenkins-hbase4:43211] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,060 INFO [RS:2;jenkins-hbase4:40867] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,060 INFO [RS:1;jenkins-hbase4:43211] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,060 INFO [RS:2;jenkins-hbase4:40867] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,060 INFO [RS:1;jenkins-hbase4:43211] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,060 INFO [RS:0;jenkins-hbase4:43281] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,061 INFO [RS:0;jenkins-hbase4:43281] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,061 INFO [RS:0;jenkins-hbase4:43281] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,078 INFO [RS:0;jenkins-hbase4:43281] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 07:54:07,078 INFO [RS:1;jenkins-hbase4:43211] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 07:54:07,078 INFO [RS:2;jenkins-hbase4:40867] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-10 07:54:07,081 INFO [RS:0;jenkins-hbase4:43281] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43281,1676015645662-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,081 INFO [RS:2;jenkins-hbase4:40867] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40867,1676015645736-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,081 INFO [RS:1;jenkins-hbase4:43211] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43211,1676015645706-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,099 INFO [RS:2;jenkins-hbase4:40867] regionserver.Replication(203): jenkins-hbase4.apache.org,40867,1676015645736 started 2023-02-10 07:54:07,099 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,40867,1676015645736, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:40867, sessionid=0x1003d3915cd0003 2023-02-10 07:54:07,099 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 07:54:07,100 DEBUG [RS:2;jenkins-hbase4:40867] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:07,100 DEBUG [RS:2;jenkins-hbase4:40867] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40867,1676015645736' 2023-02-10 07:54:07,100 DEBUG [RS:2;jenkins-hbase4:40867] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:07,100 INFO [RS:0;jenkins-hbase4:43281] regionserver.Replication(203): jenkins-hbase4.apache.org,43281,1676015645662 started 2023-02-10 07:54:07,100 INFO [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43281,1676015645662, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43281, sessionid=0x1003d3915cd0001 2023-02-10 07:54:07,100 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 07:54:07,100 DEBUG [RS:2;jenkins-hbase4:40867] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:07,101 INFO [RS:1;jenkins-hbase4:43211] regionserver.Replication(203): jenkins-hbase4.apache.org,43211,1676015645706 started 2023-02-10 07:54:07,100 DEBUG [RS:0;jenkins-hbase4:43281] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:07,101 INFO [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43211,1676015645706, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43211, sessionid=0x1003d3915cd0002 2023-02-10 07:54:07,102 DEBUG [RS:0;jenkins-hbase4:43281] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43281,1676015645662' 2023-02-10 07:54:07,102 DEBUG [RS:0;jenkins-hbase4:43281] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:07,102 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-10 07:54:07,102 DEBUG [RS:1;jenkins-hbase4:43211] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:07,103 DEBUG [RS:1;jenkins-hbase4:43211] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43211,1676015645706' 2023-02-10 07:54:07,103 DEBUG [RS:1;jenkins-hbase4:43211] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:07,102 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 07:54:07,103 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 07:54:07,103 DEBUG [RS:0;jenkins-hbase4:43281] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:07,103 DEBUG [RS:2;jenkins-hbase4:40867] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:07,103 DEBUG [RS:2;jenkins-hbase4:40867] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,40867,1676015645736' 2023-02-10 07:54:07,103 DEBUG [RS:2;jenkins-hbase4:40867] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 07:54:07,103 DEBUG [RS:1;jenkins-hbase4:43211] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:07,103 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 07:54:07,104 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 07:54:07,104 DEBUG [RS:2;jenkins-hbase4:40867] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 07:54:07,104 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-10 07:54:07,104 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-10 07:54:07,104 DEBUG [RS:1;jenkins-hbase4:43211] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:07,104 DEBUG [RS:1;jenkins-hbase4:43211] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43211,1676015645706' 2023-02-10 07:54:07,104 DEBUG [RS:1;jenkins-hbase4:43211] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 07:54:07,104 DEBUG [RS:0;jenkins-hbase4:43281] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:07,105 DEBUG [RS:0;jenkins-hbase4:43281] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43281,1676015645662' 2023-02-10 07:54:07,105 DEBUG [RS:0;jenkins-hbase4:43281] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-10 07:54:07,105 DEBUG [RS:2;jenkins-hbase4:40867] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 07:54:07,105 DEBUG [RS:1;jenkins-hbase4:43211] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 07:54:07,105 INFO [RS:2;jenkins-hbase4:40867] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 07:54:07,105 DEBUG [RS:0;jenkins-hbase4:43281] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-10 07:54:07,105 INFO [RS:2;jenkins-hbase4:40867] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 07:54:07,106 DEBUG [RS:1;jenkins-hbase4:43211] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 07:54:07,106 INFO [RS:1;jenkins-hbase4:43211] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 07:54:07,107 INFO [RS:1;jenkins-hbase4:43211] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 07:54:07,107 DEBUG [RS:0;jenkins-hbase4:43281] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-10 07:54:07,107 INFO [RS:0;jenkins-hbase4:43281] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-10 07:54:07,107 INFO [RS:0;jenkins-hbase4:43281] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-10 07:54:07,158 DEBUG [jenkins-hbase4:37109] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-10 07:54:07,162 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:07,168 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:07,168 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:07,168 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:07,168 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:07,171 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40867,1676015645736, state=OPENING 2023-02-10 07:54:07,179 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-10 07:54:07,181 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:07,182 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-10 07:54:07,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:07,216 INFO [RS:1;jenkins-hbase4:43211] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43211%2C1676015645706, suffix=, logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,43211,1676015645706, archiveDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs, maxLogs=32 2023-02-10 07:54:07,216 INFO [RS:2;jenkins-hbase4:40867] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40867%2C1676015645736, suffix=, logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,40867,1676015645736, archiveDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs, maxLogs=32 2023-02-10 07:54:07,216 INFO [RS:0;jenkins-hbase4:43281] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43281%2C1676015645662, suffix=, logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,43281,1676015645662, archiveDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs, maxLogs=32 2023-02-10 07:54:07,240 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK] 2023-02-10 07:54:07,240 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK] 2023-02-10 07:54:07,240 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK] 2023-02-10 07:54:07,247 INFO [RS:2;jenkins-hbase4:40867] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,40867,1676015645736/jenkins-hbase4.apache.org%2C40867%2C1676015645736.1676015647220 2023-02-10 07:54:07,247 DEBUG [RS:2;jenkins-hbase4:40867] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK], DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK], DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK]] 2023-02-10 07:54:07,250 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK] 2023-02-10 07:54:07,250 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK] 2023-02-10 07:54:07,250 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK] 2023-02-10 07:54:07,263 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK] 2023-02-10 07:54:07,263 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK] 2023-02-10 07:54:07,263 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK] 2023-02-10 07:54:07,269 INFO [RS:1;jenkins-hbase4:43211] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,43211,1676015645706/jenkins-hbase4.apache.org%2C43211%2C1676015645706.1676015647221 2023-02-10 07:54:07,270 DEBUG [RS:1;jenkins-hbase4:43211] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK], DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK]] 2023-02-10 07:54:07,272 INFO [RS:0;jenkins-hbase4:43281] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,43281,1676015645662/jenkins-hbase4.apache.org%2C43281%2C1676015645662.1676015647221 2023-02-10 07:54:07,276 DEBUG [RS:0;jenkins-hbase4:43281] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK], DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK]] 2023-02-10 07:54:07,369 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:07,372 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 07:54:07,375 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47048, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 07:54:07,391 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-10 07:54:07,392 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-10 07:54:07,396 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40867%2C1676015645736.meta, suffix=.meta, logDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,40867,1676015645736, archiveDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs, maxLogs=32 2023-02-10 07:54:07,418 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK] 2023-02-10 07:54:07,419 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK] 2023-02-10 07:54:07,420 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK] 2023-02-10 07:54:07,426 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/WALs/jenkins-hbase4.apache.org,40867,1676015645736/jenkins-hbase4.apache.org%2C40867%2C1676015645736.meta.1676015647398.meta 2023-02-10 07:54:07,429 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34997,DS-cdf655cd-6e55-40d8-ba50-ebd1fc93e6c4,DISK], DatanodeInfoWithStorage[127.0.0.1:38371,DS-919a5c51-ddc0-4069-8c7d-571f4c11934e,DISK], DatanodeInfoWithStorage[127.0.0.1:43743,DS-a7250a3b-d0de-4a9a-9e37-61e7518d51e0,DISK]] 2023-02-10 07:54:07,429 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:07,433 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:07,434 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-10 07:54:07,457 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-10 07:54:07,459 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-10 07:54:07,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-10 07:54:07,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:07,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-10 07:54:07,464 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-10 07:54:07,467 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-10 07:54:07,469 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/info 2023-02-10 07:54:07,470 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/info 2023-02-10 07:54:07,470 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-10 07:54:07,471 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:07,471 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-10 07:54:07,473 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/rep_barrier 2023-02-10 07:54:07,473 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/rep_barrier 2023-02-10 07:54:07,473 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-10 07:54:07,474 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:07,474 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-10 07:54:07,476 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/table 2023-02-10 07:54:07,476 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/table 2023-02-10 07:54:07,476 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-10 07:54:07,477 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:07,479 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740 2023-02-10 07:54:07,482 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740 2023-02-10 07:54:07,489 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-10 07:54:07,491 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-10 07:54:07,492 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74625465, jitterRate=0.11200608313083649}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-10 07:54:07,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676015647465Writing region info on filesystem at 1676015647465Initializing all the Stores at 1676015647467 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676015647467Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015647467Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676015647467Cleaning up temporary data from old regions at 1676015647485 (+18 ms)Cleaning up detritus from prior splits at 1676015647486 (+1 ms)Running coprocessor post-open hooks at 1676015647492 (+6 ms)Region opened successfully at 1676015647492 2023-02-10 07:54:07,504 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676015647361 2023-02-10 07:54:07,522 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-10 07:54:07,523 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-10 07:54:07,524 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,40867,1676015645736, state=OPEN 2023-02-10 07:54:07,527 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-10 07:54:07,527 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-10 07:54:07,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-10 07:54:07,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,40867,1676015645736 in 342 msec 2023-02-10 07:54:07,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-10 07:54:07,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 554 msec 2023-02-10 07:54:07,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 818 msec 2023-02-10 07:54:07,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676015645778, completionTime=-1 2023-02-10 07:54:07,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-10 07:54:07,543 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:07,612 DEBUG [hconnection-0x1154fe46-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 07:54:07,615 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47056, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 07:54:07,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:07,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015707633 2023-02-10 07:54:07,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015767633 2023-02-10 07:54:07,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 90 msec 2023-02-10 07:54:07,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37109,1676015644499-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37109,1676015644499-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37109,1676015644499-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37109, period=300000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-10 07:54:07,672 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-10 07:54:07,680 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-10 07:54:07,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-10 07:54:07,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:07,697 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:54:07,700 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:54:07,724 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:07,727 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c empty. 2023-02-10 07:54:07,728 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:07,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-10 07:54:07,768 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:07,770 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => a50233cec7373b84b61f6ef56d6ed02c, NAME => 'hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:54:07,789 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:07,789 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing a50233cec7373b84b61f6ef56d6ed02c, disabling compactions & flushes 2023-02-10 07:54:07,789 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:54:07,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:54:07,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. after waiting 0 ms 2023-02-10 07:54:07,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:54:07,790 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:54:07,790 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for a50233cec7373b84b61f6ef56d6ed02c: Waiting for close lock at 1676015647789Disabling compacts and flushes for region at 1676015647789Disabling writes for close at 1676015647790 (+1 ms)Writing region close event to WAL at 1676015647790Closed at 1676015647790 2023-02-10 07:54:07,796 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:54:07,812 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676015647799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015647799"}]},"ts":"1676015647799"} 2023-02-10 07:54:07,839 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:54:07,841 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:54:07,846 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015647842"}]},"ts":"1676015647842"} 2023-02-10 07:54:07,853 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-10 07:54:07,858 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:07,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:07,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:07,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:07,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:07,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a50233cec7373b84b61f6ef56d6ed02c, ASSIGN}] 2023-02-10 07:54:07,866 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=a50233cec7373b84b61f6ef56d6ed02c, ASSIGN 2023-02-10 07:54:07,868 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=a50233cec7373b84b61f6ef56d6ed02c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:08,020 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:08,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:08,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676015648021"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015648021"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015648021"}]},"ts":"1676015648021"} 2023-02-10 07:54:08,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure a50233cec7373b84b61f6ef56d6ed02c, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:08,190 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:54:08,192 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a50233cec7373b84b61f6ef56d6ed02c, NAME => 'hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:08,193 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:08,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:08,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:08,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:08,194 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:08,197 INFO [StoreOpener-a50233cec7373b84b61f6ef56d6ed02c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:08,201 DEBUG [StoreOpener-a50233cec7373b84b61f6ef56d6ed02c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/info 2023-02-10 07:54:08,201 DEBUG [StoreOpener-a50233cec7373b84b61f6ef56d6ed02c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/info 2023-02-10 07:54:08,201 INFO [StoreOpener-a50233cec7373b84b61f6ef56d6ed02c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a50233cec7373b84b61f6ef56d6ed02c columnFamilyName info 2023-02-10 07:54:08,202 INFO [StoreOpener-a50233cec7373b84b61f6ef56d6ed02c-1] regionserver.HStore(310): Store=a50233cec7373b84b61f6ef56d6ed02c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:08,207 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:08,209 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:08,215 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:54:08,219 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:08,220 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a50233cec7373b84b61f6ef56d6ed02c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62894995, jitterRate=-0.06279154121875763}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 07:54:08,220 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a50233cec7373b84b61f6ef56d6ed02c: Running coprocessor pre-open hook at 1676015648195Writing region info on filesystem at 1676015648195Initializing all the Stores at 1676015648197 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676015648197Cleaning up temporary data from old regions at 1676015648210 (+13 ms)Cleaning up detritus from prior splits at 1676015648210Running coprocessor post-open hooks at 1676015648220 (+10 ms)Region opened successfully at 1676015648220 2023-02-10 07:54:08,224 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c., pid=6, masterSystemTime=1676015648180 2023-02-10 07:54:08,229 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:54:08,229 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:54:08,231 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:08,232 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676015648230"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015648230"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015648230"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015648230"}]},"ts":"1676015648230"} 2023-02-10 07:54:08,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-10 07:54:08,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure a50233cec7373b84b61f6ef56d6ed02c, server=jenkins-hbase4.apache.org,40867,1676015645736 in 210 msec 2023-02-10 07:54:08,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-10 07:54:08,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=a50233cec7373b84b61f6ef56d6ed02c, ASSIGN in 380 msec 2023-02-10 07:54:08,249 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:54:08,250 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015648250"}]},"ts":"1676015648250"} 2023-02-10 07:54:08,253 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-10 07:54:08,258 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:54:08,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 575 msec 2023-02-10 07:54:08,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-10 07:54:08,299 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-10 07:54:08,299 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:08,343 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:08,368 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-10 07:54:08,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 42 msec 2023-02-10 07:54:08,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:08,405 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-10 07:54:08,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 25 msec 2023-02-10 07:54:08,432 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-10 07:54:08,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-10 07:54:08,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.644sec 2023-02-10 07:54:08,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-10 07:54:08,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-10 07:54:08,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-10 07:54:08,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37109,1676015644499-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-10 07:54:08,447 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37109,1676015644499-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-10 07:54:08,463 DEBUG [Listener at localhost/43795] zookeeper.ReadOnlyZKClient(139): Connect 0x37e887de to 127.0.0.1:60412 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-10 07:54:08,480 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-10 07:54:08,482 DEBUG [Listener at localhost/43795] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c8d5aec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-10 07:54:08,504 DEBUG [hconnection-0x7e2f5f25-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 07:54:08,520 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47068, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 07:54:08,531 INFO [Listener at localhost/43795] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:54:08,562 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=357, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=284, ProcessCount=168, AvailableMemoryMB=3805 2023-02-10 07:54:08,567 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:08,577 DEBUG [Listener at localhost/43795] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-10 07:54:08,581 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44470, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-10 07:54:08,596 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:54:08,596 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:54:08,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:54:08,604 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:08,607 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:54:08,609 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:54:08,613 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:08,614 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d empty. 2023-02-10 07:54:08,618 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:08,618 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-10 07:54:08,662 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:08,664 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1d79b8ea1a81e86b36c3306b6c1de43d, NAME => 'testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:54:08,694 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:08,695 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 1d79b8ea1a81e86b36c3306b6c1de43d, disabling compactions & flushes 2023-02-10 07:54:08,695 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:08,695 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:08,695 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. after waiting 0 ms 2023-02-10 07:54:08,695 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:08,695 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:08,695 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 1d79b8ea1a81e86b36c3306b6c1de43d: Waiting for close lock at 1676015648695Disabling compacts and flushes for region at 1676015648695Disabling writes for close at 1676015648695Writing region close event to WAL at 1676015648695Closed at 1676015648695 2023-02-10 07:54:08,700 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:54:08,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676015648702"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015648702"}]},"ts":"1676015648702"} 2023-02-10 07:54:08,706 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:54:08,708 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:54:08,708 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015648708"}]},"ts":"1676015648708"} 2023-02-10 07:54:08,711 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-10 07:54:08,717 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:08,720 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:08,720 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:08,720 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:08,720 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:08,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d79b8ea1a81e86b36c3306b6c1de43d, ASSIGN}] 2023-02-10 07:54:08,727 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d79b8ea1a81e86b36c3306b6c1de43d, ASSIGN 2023-02-10 07:54:08,728 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d79b8ea1a81e86b36c3306b6c1de43d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:08,879 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:08,880 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1d79b8ea1a81e86b36c3306b6c1de43d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:08,880 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676015648880"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015648880"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015648880"}]},"ts":"1676015648880"} 2023-02-10 07:54:08,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 1d79b8ea1a81e86b36c3306b6c1de43d, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:09,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1d79b8ea1a81e86b36c3306b6c1de43d, NAME => 'testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:09,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:09,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:09,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,048 INFO [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,050 DEBUG [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1 2023-02-10 07:54:09,051 DEBUG [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1 2023-02-10 07:54:09,051 INFO [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d79b8ea1a81e86b36c3306b6c1de43d columnFamilyName cf1 2023-02-10 07:54:09,052 INFO [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] regionserver.HStore(310): Store=1d79b8ea1a81e86b36c3306b6c1de43d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:09,052 INFO [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,055 DEBUG [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2 2023-02-10 07:54:09,055 DEBUG [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2 2023-02-10 07:54:09,055 INFO [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1d79b8ea1a81e86b36c3306b6c1de43d columnFamilyName cf2 2023-02-10 07:54:09,056 INFO [StoreOpener-1d79b8ea1a81e86b36c3306b6c1de43d-1] regionserver.HStore(310): Store=1d79b8ea1a81e86b36c3306b6c1de43d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:09,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:09,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:09,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:09,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1d79b8ea1a81e86b36c3306b6c1de43d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68695225, jitterRate=0.023638620972633362}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:09,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1d79b8ea1a81e86b36c3306b6c1de43d: Running coprocessor pre-open hook at 1676015649045Writing region info on filesystem at 1676015649045Initializing all the Stores at 1676015649047 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015649047Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015649048 (+1 ms)Cleaning up temporary data from old regions at 1676015649061 (+13 ms)Cleaning up detritus from prior splits at 1676015649062 (+1 ms)Running coprocessor post-open hooks at 1676015649072 (+10 ms)Region opened successfully at 1676015649072 2023-02-10 07:54:09,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d., pid=11, masterSystemTime=1676015649037 2023-02-10 07:54:09,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,078 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1d79b8ea1a81e86b36c3306b6c1de43d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676015649078"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015649078"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015649078"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015649078"}]},"ts":"1676015649078"} 2023-02-10 07:54:09,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-10 07:54:09,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 1d79b8ea1a81e86b36c3306b6c1de43d, server=jenkins-hbase4.apache.org,40867,1676015645736 in 199 msec 2023-02-10 07:54:09,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-10 07:54:09,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d79b8ea1a81e86b36c3306b6c1de43d, ASSIGN in 365 msec 2023-02-10 07:54:09,090 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:54:09,090 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015649090"}]},"ts":"1676015649090"} 2023-02-10 07:54:09,092 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-10 07:54:09,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:54:09,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 493 msec 2023-02-10 07:54:09,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:09,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,168 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:09,168 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,169 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-10 07:54:09,169 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-10 07:54:09,169 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,169 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,171 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,171 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,171 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:09,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,175 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,175 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:09,177 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,177 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. started... 2023-02-10 07:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:09,179 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1d79b8ea1a81e86b36c3306b6c1de43d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:54:09,179 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,181 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,181 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,181 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,182 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,182 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,252 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/6e8f8350acc147d79c8d787ef19cfbc6 2023-02-10 07:54:09,336 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/4e3a8e258fc6440d844068a50c1893fe 2023-02-10 07:54:09,346 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/6e8f8350acc147d79c8d787ef19cfbc6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6 2023-02-10 07:54:09,359 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:09,363 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/4e3a8e258fc6440d844068a50c1893fe as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe 2023-02-10 07:54:09,380 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:09,383 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 204ms, sequenceid=6, compaction requested=false 2023-02-10 07:54:09,384 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:09,386 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1d79b8ea1a81e86b36c3306b6c1de43d: 2023-02-10 07:54:09,386 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,386 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:09,389 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:09,389 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,389 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,389 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,391 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,391 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,392 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,392 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-10 07:54:09,395 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:09,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d61269d[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:09,395 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:09,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:09,400 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:09,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,401 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,401 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,401 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:09,405 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,405 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,405 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,405 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,406 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,406 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,406 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,406 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,406 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,407 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:09,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:09,412 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-10 07:54:09,412 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:09,412 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:09,412 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,412 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:09,414 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,414 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,415 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,414 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,415 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,416 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,417 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,419 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,420 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,420 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,432 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:09,432 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:09,431 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,432 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,432 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,433 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,433 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,433 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,433 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,433 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,433 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,435 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:09,435 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:09,535 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:09,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:09,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:09,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:09,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,568 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:09,568 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,569 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-10 07:54:09,569 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-10 07:54:09,570 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,570 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,572 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,572 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,572 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,572 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,572 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,573 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:09,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:09,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:09,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,575 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,575 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,575 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,575 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:09,577 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,577 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. started... 2023-02-10 07:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,577 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,578 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1d79b8ea1a81e86b36c3306b6c1de43d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:09,579 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,580 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,581 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,600 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/5ec98ae4e11e411ea43598a99978c0dd 2023-02-10 07:54:09,627 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/9fe467b28a0d4ebb834127555eaf01d7 2023-02-10 07:54:09,636 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/5ec98ae4e11e411ea43598a99978c0dd as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd 2023-02-10 07:54:09,644 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:09,645 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/9fe467b28a0d4ebb834127555eaf01d7 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7 2023-02-10 07:54:09,654 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:09,655 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 78ms, sequenceid=19, compaction requested=false 2023-02-10 07:54:09,655 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1d79b8ea1a81e86b36c3306b6c1de43d: 2023-02-10 07:54:09,655 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,655 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:09,656 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:09,656 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,656 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,656 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,658 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,658 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,658 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,658 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-10 07:54:09,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@281c952c[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:09,661 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:09,661 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,663 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,663 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:09,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:09,666 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,666 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,666 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,666 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,667 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,667 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,667 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,667 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,667 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,667 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:09,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:09,672 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-10 07:54:09,672 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:09,672 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:09,672 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,672 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:09,674 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,674 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,674 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,674 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,674 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,674 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,675 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,675 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,676 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,677 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,680 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,681 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,681 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,695 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,695 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,696 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,696 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,696 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,696 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,696 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:09,697 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:09,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,696 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,696 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,697 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:09,698 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:09,698 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,698 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,698 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,698 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,698 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,698 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,699 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,699 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,798 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:09,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:09,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:09,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:09,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,829 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:09,830 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,830 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-10 07:54:09,830 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-10 07:54:09,831 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,831 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,832 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,832 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,832 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,832 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,833 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,833 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:09,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,835 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:09,835 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,835 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,835 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,835 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:09,836 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:09,836 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,836 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,836 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,836 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:09,836 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,836 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:09,837 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,837 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:09,838 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:09,838 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,838 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. started... 2023-02-10 07:54:09,838 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1d79b8ea1a81e86b36c3306b6c1de43d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:09,839 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,840 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,840 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,840 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,841 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,841 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,858 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8 2023-02-10 07:54:09,883 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/57f2feb94820493e92e74839618be3e0 2023-02-10 07:54:09,892 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8 2023-02-10 07:54:09,899 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:09,900 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/57f2feb94820493e92e74839618be3e0 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0 2023-02-10 07:54:09,907 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:09,909 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 71ms, sequenceid=32, compaction requested=true 2023-02-10 07:54:09,909 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1d79b8ea1a81e86b36c3306b6c1de43d: 2023-02-10 07:54:09,909 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:09,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:09,910 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:09,910 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,910 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:09,911 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:09,913 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,913 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,914 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,914 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-10 07:54:09,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6aa1a534[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:09,917 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:09,917 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,918 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,918 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,918 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,918 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,918 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:09,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,918 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,919 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:09,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:09,923 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,923 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,923 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,924 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,924 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,924 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,924 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:09,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:09,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:09,931 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-10 07:54:09,931 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:09,931 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:09,931 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,931 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:09,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:09,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,934 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,935 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:09,940 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,941 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,941 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,944 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,944 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,945 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,945 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,945 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,945 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:09,945 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,945 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,945 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:09,945 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,946 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:09,946 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,945 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:09,946 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:09,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,946 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:09,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:09,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,946 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:09,947 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:09,947 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:09,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:09,947 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:09,947 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:09,947 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,947 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:09,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:09,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,047 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:10,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:10,070 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:10,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:10,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,074 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:10,074 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,075 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-10 07:54:10,075 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-10 07:54:10,075 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,075 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,077 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,077 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,077 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,077 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,077 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,077 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,079 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,079 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,079 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,079 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,079 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,080 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,080 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,080 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,080 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,080 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,080 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,080 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,080 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:10,080 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:10,080 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,080 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. started... 2023-02-10 07:54:10,081 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1d79b8ea1a81e86b36c3306b6c1de43d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:10,084 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,084 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,084 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,084 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,084 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,084 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,087 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,087 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,087 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,087 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,087 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,088 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,105 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/dba4b4c800ea46928eba936e6728eb03 2023-02-10 07:54:10,130 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/e8122b24b60849508a9d4d75afe25175 2023-02-10 07:54:10,138 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/dba4b4c800ea46928eba936e6728eb03 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03 2023-02-10 07:54:10,146 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:10,147 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/e8122b24b60849508a9d4d75afe25175 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175 2023-02-10 07:54:10,155 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:10,156 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 76ms, sequenceid=45, compaction requested=true 2023-02-10 07:54:10,157 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1d79b8ea1a81e86b36c3306b6c1de43d: 2023-02-10 07:54:10,157 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:10,157 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,158 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,158 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,162 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,162 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,162 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,162 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-10 07:54:10,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@334955cd[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:10,165 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:10,165 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,167 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,167 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,167 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:10,166 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:10,166 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,167 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:10,167 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:10,169 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,169 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,169 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,169 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,170 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,170 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,170 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,170 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,170 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,171 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:10,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:10,175 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-10 07:54:10,175 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:10,175 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:10,175 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,175 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:10,176 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,178 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,178 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,180 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,180 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,181 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,184 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,185 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:10,185 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:10,185 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,185 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,186 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:10,186 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,186 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,186 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:10,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,186 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,187 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,187 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,187 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,187 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,287 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:10,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:10,312 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:10,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:10,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,314 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:10,314 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,315 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-10 07:54:10,315 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-10 07:54:10,316 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,316 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,317 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,317 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,317 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,317 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,318 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,318 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:10,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,321 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,321 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,321 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,321 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,323 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,322 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:10,323 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,323 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:10,323 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. started... 2023-02-10 07:54:10,323 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,323 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,323 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,323 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1d79b8ea1a81e86b36c3306b6c1de43d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:10,324 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,325 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,325 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,326 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,327 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,327 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,327 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,327 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,327 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,344 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/b29cf93200c64490bc731d69520561a0 2023-02-10 07:54:10,391 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/22a2cc8626284278960c01b2caa18ae9 2023-02-10 07:54:10,399 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/b29cf93200c64490bc731d69520561a0 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/b29cf93200c64490bc731d69520561a0 2023-02-10 07:54:10,406 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/b29cf93200c64490bc731d69520561a0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:10,407 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/22a2cc8626284278960c01b2caa18ae9 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/22a2cc8626284278960c01b2caa18ae9 2023-02-10 07:54:10,414 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/22a2cc8626284278960c01b2caa18ae9, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:10,415 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 92ms, sequenceid=58, compaction requested=true 2023-02-10 07:54:10,415 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1d79b8ea1a81e86b36c3306b6c1de43d: 2023-02-10 07:54:10,415 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,415 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:10,416 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:10,416 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,416 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,416 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,419 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,419 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,419 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,419 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-10 07:54:10,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17b8c8f7[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:10,421 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:10,421 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,422 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,422 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,422 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,423 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,423 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:10,423 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:10,425 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,425 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,425 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,426 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,426 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,426 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,426 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,426 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,426 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,426 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:10,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:10,431 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-10 07:54:10,431 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:10,431 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:10,431 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,431 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:10,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,435 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,434 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,435 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,435 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,437 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,438 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,438 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,443 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,444 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,444 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:10,444 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,445 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:10,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,445 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,445 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:10,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,445 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,445 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,445 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,445 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,545 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:10,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:10,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:10,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:10,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,571 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:10,571 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,571 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-10 07:54:10,572 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-10 07:54:10,572 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,572 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,573 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,573 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,573 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,573 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,574 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,574 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-10 07:54:10,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-10 07:54:10,577 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,577 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,577 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-10 07:54:10,578 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,579 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. started... 2023-02-10 07:54:10,579 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,579 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1d79b8ea1a81e86b36c3306b6c1de43d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:10,580 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,582 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,582 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,582 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,583 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,583 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,596 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/c0a3ff9ba9204500912ed1612cc4ef8a 2023-02-10 07:54:10,614 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/143b87ef94b3494ead279aac76b8f455 2023-02-10 07:54:10,622 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/c0a3ff9ba9204500912ed1612cc4ef8a as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/c0a3ff9ba9204500912ed1612cc4ef8a 2023-02-10 07:54:10,629 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/c0a3ff9ba9204500912ed1612cc4ef8a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:10,631 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/143b87ef94b3494ead279aac76b8f455 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/143b87ef94b3494ead279aac76b8f455 2023-02-10 07:54:10,638 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/143b87ef94b3494ead279aac76b8f455, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:10,639 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 60ms, sequenceid=71, compaction requested=true 2023-02-10 07:54:10,639 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1d79b8ea1a81e86b36c3306b6c1de43d: 2023-02-10 07:54:10,639 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,639 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:10,640 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:10,640 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,640 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-10 07:54:10,640 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-10 07:54:10,642 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,642 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,642 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:10,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-10 07:54:10,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b419ad9[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:10,644 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:10,644 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,646 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:10,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:10,646 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:10,648 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,648 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,648 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,648 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,648 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,648 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,650 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-10 07:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,650 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-10 07:54:10,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:10,654 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-10 07:54:10,654 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:10,654 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:10,654 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,654 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:10,655 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,655 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,655 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,655 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,655 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,655 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,656 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,656 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,657 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,657 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,659 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:10,660 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,660 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,663 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-10 07:54:10,663 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:10,663 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:10,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:10,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,664 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:10,664 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:10,664 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,663 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:10,664 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,664 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:10,665 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:10,665 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,665 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:10,665 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,665 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:10,665 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-10 07:54:10,765 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:10,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:10,789 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,797 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:10,799 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:10,799 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:10,799 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:10,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d79b8ea1a81e86b36c3306b6c1de43d, UNASSIGN}] 2023-02-10 07:54:10,809 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d79b8ea1a81e86b36c3306b6c1de43d, UNASSIGN 2023-02-10 07:54:10,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1d79b8ea1a81e86b36c3306b6c1de43d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:10,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676015650810"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015650810"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015650810"}]},"ts":"1676015650810"} 2023-02-10 07:54:10,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 1d79b8ea1a81e86b36c3306b6c1de43d, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:10,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:10,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1d79b8ea1a81e86b36c3306b6c1de43d, disabling compactions & flushes 2023-02-10 07:54:10,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. after waiting 0 ms 2023-02-10 07:54:10,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:10,973 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1d79b8ea1a81e86b36c3306b6c1de43d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-10 07:54:10,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/89cad5d656774a0e9a88bad5e3824190 2023-02-10 07:54:11,011 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/d38d8195a249477eaf5c246a883a89b1 2023-02-10 07:54:11,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf1/89cad5d656774a0e9a88bad5e3824190 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/89cad5d656774a0e9a88bad5e3824190 2023-02-10 07:54:11,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/89cad5d656774a0e9a88bad5e3824190, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:54:11,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.tmp/cf2/d38d8195a249477eaf5c246a883a89b1 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/d38d8195a249477eaf5c246a883a89b1 2023-02-10 07:54:11,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/d38d8195a249477eaf5c246a883a89b1, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:54:11,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 64ms, sequenceid=82, compaction requested=true 2023-02-10 07:54:11,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-10 07:54:11,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:11,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. 2023-02-10 07:54:11,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1d79b8ea1a81e86b36c3306b6c1de43d: Waiting for close lock at 1676015650973Running coprocessor pre-close hooks at 1676015650973Disabling compacts and flushes for region at 1676015650973Disabling writes for close at 1676015650973Obtaining lock to block concurrent updates at 1676015650973Preparing flush snapshotting stores in 1d79b8ea1a81e86b36c3306b6c1de43d at 1676015650973Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676015650973Flushing stores of testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. at 1676015650974 (+1 ms)Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf1: creating writer at 1676015650974Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf1: appending metadata at 1676015650977 (+3 ms)Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf1: closing flushed file at 1676015650978 (+1 ms)Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf2: creating writer at 1676015650997 (+19 ms)Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf2: appending metadata at 1676015650999 (+2 ms)Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf2: closing flushed file at 1676015650999Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf1: reopening flushed file at 1676015651019 (+20 ms)Flushing 1d79b8ea1a81e86b36c3306b6c1de43d/cf2: reopening flushed file at 1676015651030 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1d79b8ea1a81e86b36c3306b6c1de43d in 64ms, sequenceid=82, compaction requested=true at 1676015651037 (+7 ms)Writing region close event to WAL at 1676015651053 (+16 ms)Running coprocessor post-close hooks at 1676015651058 (+5 ms)Closed at 1676015651059 (+1 ms) 2023-02-10 07:54:11,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,062 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=1d79b8ea1a81e86b36c3306b6c1de43d, regionState=CLOSED 2023-02-10 07:54:11,063 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676015651062"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015651062"}]},"ts":"1676015651062"} 2023-02-10 07:54:11,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-10 07:54:11,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 1d79b8ea1a81e86b36c3306b6c1de43d, server=jenkins-hbase4.apache.org,40867,1676015645736 in 253 msec 2023-02-10 07:54:11,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-10 07:54:11,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=1d79b8ea1a81e86b36c3306b6c1de43d, UNASSIGN in 260 msec 2023-02-10 07:54:11,088 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=1d79b8ea1a81e86b36c3306b6c1de43d, threads=14 2023-02-10 07:54:11,089 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/143b87ef94b3494ead279aac76b8f455 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,089 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/22a2cc8626284278960c01b2caa18ae9 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,089 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,089 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,089 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,090 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/d38d8195a249477eaf5c246a883a89b1 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,090 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,090 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,090 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/89cad5d656774a0e9a88bad5e3824190 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,091 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/b29cf93200c64490bc731d69520561a0 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,091 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,091 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,091 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/c0a3ff9ba9204500912ed1612cc4ef8a for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,091 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,126 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/d38d8195a249477eaf5c246a883a89b1, top=true 2023-02-10 07:54:11,129 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/143b87ef94b3494ead279aac76b8f455, top=true 2023-02-10 07:54:11,138 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/b29cf93200c64490bc731d69520561a0, top=true 2023-02-10 07:54:11,139 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/89cad5d656774a0e9a88bad5e3824190, top=true 2023-02-10 07:54:11,140 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/22a2cc8626284278960c01b2caa18ae9, top=true 2023-02-10 07:54:11,150 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/c0a3ff9ba9204500912ed1612cc4ef8a, top=true 2023-02-10 07:54:11,185 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.splits/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-d38d8195a249477eaf5c246a883a89b1 for child: a5fd8742bb346a85e0d5f561d7ce5673, parent: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,186 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/d38d8195a249477eaf5c246a883a89b1 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,204 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.splits/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-143b87ef94b3494ead279aac76b8f455 for child: a5fd8742bb346a85e0d5f561d7ce5673, parent: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,204 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/143b87ef94b3494ead279aac76b8f455 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,208 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.splits/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-c0a3ff9ba9204500912ed1612cc4ef8a for child: a5fd8742bb346a85e0d5f561d7ce5673, parent: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,208 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/c0a3ff9ba9204500912ed1612cc4ef8a for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,209 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.splits/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-b29cf93200c64490bc731d69520561a0 for child: a5fd8742bb346a85e0d5f561d7ce5673, parent: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,209 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/b29cf93200c64490bc731d69520561a0 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,209 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.splits/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-22a2cc8626284278960c01b2caa18ae9 for child: a5fd8742bb346a85e0d5f561d7ce5673, parent: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,209 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/22a2cc8626284278960c01b2caa18ae9 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,212 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/.splits/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-89cad5d656774a0e9a88bad5e3824190 for child: a5fd8742bb346a85e0d5f561d7ce5673, parent: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,212 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/89cad5d656774a0e9a88bad5e3824190 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,293 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,302 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,304 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,305 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,308 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,312 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,312 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,677 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6 for region: 1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:11,677 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 1d79b8ea1a81e86b36c3306b6c1de43d Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:54:11,730 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:54:11,732 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:54:11,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676015651735"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676015651735"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676015651735"}]},"ts":"1676015651735"} 2023-02-10 07:54:11,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015651735"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015651735"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015651735"}]},"ts":"1676015651735"} 2023-02-10 07:54:11,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015651735"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015651735"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015651735"}]},"ts":"1676015651735"} 2023-02-10 07:54:11,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=817f15842cc8547f342051dd23a15f24, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a5fd8742bb346a85e0d5f561d7ce5673, ASSIGN}] 2023-02-10 07:54:11,790 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a5fd8742bb346a85e0d5f561d7ce5673, ASSIGN 2023-02-10 07:54:11,791 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a5fd8742bb346a85e0d5f561d7ce5673, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:11,792 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=817f15842cc8547f342051dd23a15f24, ASSIGN 2023-02-10 07:54:11,793 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=817f15842cc8547f342051dd23a15f24, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:11,941 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-10 07:54:11,943 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a5fd8742bb346a85e0d5f561d7ce5673, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:11,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015651943"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015651943"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015651943"}]},"ts":"1676015651943"} 2023-02-10 07:54:11,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure a5fd8742bb346a85e0d5f561d7ce5673, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:11,950 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=817f15842cc8547f342051dd23a15f24, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:11,950 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015651950"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015651950"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015651950"}]},"ts":"1676015651950"} 2023-02-10 07:54:11,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 817f15842cc8547f342051dd23a15f24, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:12,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 817f15842cc8547f342051dd23a15f24, NAME => 'testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.', STARTKEY => '', ENDKEY => '26'} 2023-02-10 07:54:12,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:12,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:12,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,106 INFO [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,107 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1 2023-02-10 07:54:12,107 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1 2023-02-10 07:54:12,108 INFO [StoreOpener-817f15842cc8547f342051dd23a15f24-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 817f15842cc8547f342051dd23a15f24 columnFamilyName cf1 2023-02-10 07:54:12,140 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd-bottom 2023-02-10 07:54:12,147 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6-bottom 2023-02-10 07:54:12,154 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8-bottom 2023-02-10 07:54:12,161 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03-bottom 2023-02-10 07:54:12,162 INFO [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(310): Store=817f15842cc8547f342051dd23a15f24/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:12,162 INFO [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,163 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2 2023-02-10 07:54:12,163 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2 2023-02-10 07:54:12,164 INFO [StoreOpener-817f15842cc8547f342051dd23a15f24-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 817f15842cc8547f342051dd23a15f24 columnFamilyName cf2 2023-02-10 07:54:12,180 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe-bottom 2023-02-10 07:54:12,186 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0-bottom 2023-02-10 07:54:12,193 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7-bottom 2023-02-10 07:54:12,200 DEBUG [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175-bottom 2023-02-10 07:54:12,200 INFO [StoreOpener-817f15842cc8547f342051dd23a15f24-1] regionserver.HStore(310): Store=817f15842cc8547f342051dd23a15f24/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:12,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:12,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 817f15842cc8547f342051dd23a15f24; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65844051, jitterRate=-0.018847182393074036}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:12,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 817f15842cc8547f342051dd23a15f24: Running coprocessor pre-open hook at 1676015652105Writing region info on filesystem at 1676015652105Initializing all the Stores at 1676015652106 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015652106Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015652106Cleaning up temporary data from old regions at 1676015652204 (+98 ms)Cleaning up detritus from prior splits at 1676015652204Running coprocessor post-open hooks at 1676015652207 (+3 ms)Region opened successfully at 1676015652207 2023-02-10 07:54:12,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24., pid=18, masterSystemTime=1676015652098 2023-02-10 07:54:12,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:12,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:12,211 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-10 07:54:12,211 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-10 07:54:12,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,214 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,214 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1912): 817f15842cc8547f342051dd23a15f24/cf1 is initiating minor compaction (all files) 2023-02-10 07:54:12,214 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,214 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1912): 817f15842cc8547f342051dd23a15f24/cf2 is initiating minor compaction (all files) 2023-02-10 07:54:12,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,214 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 817f15842cc8547f342051dd23a15f24/cf2 in testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a5fd8742bb346a85e0d5f561d7ce5673, NAME => 'testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.', STARTKEY => '26', ENDKEY => ''} 2023-02-10 07:54:12,214 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 817f15842cc8547f342051dd23a15f24/cf1 in testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,215 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=817f15842cc8547f342051dd23a15f24, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:12,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:12,215 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/.tmp, totalSize=19.9 K 2023-02-10 07:54:12,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,215 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015652215"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015652215"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015652215"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015652215"}]},"ts":"1676015652215"} 2023-02-10 07:54:12,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:12,215 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/.tmp, totalSize=19.9 K 2023-02-10 07:54:12,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,218 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015649138 2023-02-10 07:54:12,218 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015649138 2023-02-10 07:54:12,219 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015649543 2023-02-10 07:54:12,219 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015649543 2023-02-10 07:54:12,220 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015649801 2023-02-10 07:54:12,220 INFO [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,220 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015649801 2023-02-10 07:54:12,221 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015650050 2023-02-10 07:54:12,221 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015650050 2023-02-10 07:54:12,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-10 07:54:12,223 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1 2023-02-10 07:54:12,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 817f15842cc8547f342051dd23a15f24, server=jenkins-hbase4.apache.org,40867,1676015645736 in 267 msec 2023-02-10 07:54:12,223 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1 2023-02-10 07:54:12,223 INFO [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5fd8742bb346a85e0d5f561d7ce5673 columnFamilyName cf1 2023-02-10 07:54:12,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=817f15842cc8547f342051dd23a15f24, ASSIGN in 434 msec 2023-02-10 07:54:12,241 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] throttle.PressureAwareThroughputController(145): 817f15842cc8547f342051dd23a15f24#cf1#compaction#14 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:12,241 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] throttle.PressureAwareThroughputController(145): 817f15842cc8547f342051dd23a15f24#cf2#compaction#15 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:12,244 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd-top 2023-02-10 07:54:12,249 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6-top 2023-02-10 07:54:12,254 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8-top 2023-02-10 07:54:12,259 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03-top 2023-02-10 07:54:12,266 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-89cad5d656774a0e9a88bad5e3824190 2023-02-10 07:54:12,274 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-b29cf93200c64490bc731d69520561a0 2023-02-10 07:54:12,281 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/.tmp/cf2/25721165ec7d45e2b5093d1781f0ebb9 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/25721165ec7d45e2b5093d1781f0ebb9 2023-02-10 07:54:12,282 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-c0a3ff9ba9204500912ed1612cc4ef8a 2023-02-10 07:54:12,283 INFO [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(310): Store=a5fd8742bb346a85e0d5f561d7ce5673/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:12,283 INFO [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,284 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2 2023-02-10 07:54:12,284 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2 2023-02-10 07:54:12,285 INFO [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5fd8742bb346a85e0d5f561d7ce5673 columnFamilyName cf2 2023-02-10 07:54:12,288 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/.tmp/cf1/ea4d6822a9cb4157b90b116235070688 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/ea4d6822a9cb4157b90b116235070688 2023-02-10 07:54:12,303 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 817f15842cc8547f342051dd23a15f24/cf2 of 817f15842cc8547f342051dd23a15f24 into 25721165ec7d45e2b5093d1781f0ebb9(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:12,305 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 817f15842cc8547f342051dd23a15f24/cf1 of 817f15842cc8547f342051dd23a15f24 into ea4d6822a9cb4157b90b116235070688(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:12,309 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 817f15842cc8547f342051dd23a15f24: 2023-02-10 07:54:12,309 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 817f15842cc8547f342051dd23a15f24: 2023-02-10 07:54:12,309 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24., storeName=817f15842cc8547f342051dd23a15f24/cf1, priority=12, startTime=1676015652210; duration=0sec 2023-02-10 07:54:12,309 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24., storeName=817f15842cc8547f342051dd23a15f24/cf2, priority=12, startTime=1676015652210; duration=0sec 2023-02-10 07:54:12,310 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:12,310 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:12,314 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe-top 2023-02-10 07:54:12,320 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0-top 2023-02-10 07:54:12,325 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7-top 2023-02-10 07:54:12,331 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175-top 2023-02-10 07:54:12,336 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-143b87ef94b3494ead279aac76b8f455 2023-02-10 07:54:12,341 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-22a2cc8626284278960c01b2caa18ae9 2023-02-10 07:54:12,346 DEBUG [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-d38d8195a249477eaf5c246a883a89b1 2023-02-10 07:54:12,347 INFO [StoreOpener-a5fd8742bb346a85e0d5f561d7ce5673-1] regionserver.HStore(310): Store=a5fd8742bb346a85e0d5f561d7ce5673/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:12,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:12,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a5fd8742bb346a85e0d5f561d7ce5673; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60938157, jitterRate=-0.09195069968700409}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:12,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a5fd8742bb346a85e0d5f561d7ce5673: Running coprocessor pre-open hook at 1676015652217Writing region info on filesystem at 1676015652217Initializing all the Stores at 1676015652219 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015652219Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015652220 (+1 ms)Cleaning up temporary data from old regions at 1676015652350 (+130 ms)Cleaning up detritus from prior splits at 1676015652351 (+1 ms)Running coprocessor post-open hooks at 1676015652355 (+4 ms)Region opened successfully at 1676015652355 2023-02-10 07:54:12,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673., pid=17, masterSystemTime=1676015652098 2023-02-10 07:54:12,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:12,357 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:12,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:12,358 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:12,358 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,358 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1912): a5fd8742bb346a85e0d5f561d7ce5673/cf1 is initiating minor compaction (all files) 2023-02-10 07:54:12,358 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,358 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a5fd8742bb346a85e0d5f561d7ce5673/cf1 in testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,358 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1912): a5fd8742bb346a85e0d5f561d7ce5673/cf2 is initiating minor compaction (all files) 2023-02-10 07:54:12,358 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a5fd8742bb346a85e0d5f561d7ce5673/cf2 in testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,359 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-b29cf93200c64490bc731d69520561a0, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-c0a3ff9ba9204500912ed1612cc4ef8a, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-89cad5d656774a0e9a88bad5e3824190] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/.tmp, totalSize=34.9 K 2023-02-10 07:54:12,359 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-22a2cc8626284278960c01b2caa18ae9, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-143b87ef94b3494ead279aac76b8f455, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-d38d8195a249477eaf5c246a883a89b1] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/.tmp, totalSize=34.9 K 2023-02-10 07:54:12,359 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015649138 2023-02-10 07:54:12,362 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015649138 2023-02-10 07:54:12,362 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015649543 2023-02-10 07:54:12,362 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015649543 2023-02-10 07:54:12,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,363 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015649801 2023-02-10 07:54:12,364 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015649801 2023-02-10 07:54:12,364 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=a5fd8742bb346a85e0d5f561d7ce5673, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:12,364 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015652364"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015652364"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015652364"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015652364"}]},"ts":"1676015652364"} 2023-02-10 07:54:12,365 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015650050 2023-02-10 07:54:12,365 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015650050 2023-02-10 07:54:12,366 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-22a2cc8626284278960c01b2caa18ae9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015650289 2023-02-10 07:54:12,366 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-143b87ef94b3494ead279aac76b8f455, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015650548 2023-02-10 07:54:12,366 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-b29cf93200c64490bc731d69520561a0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015650289 2023-02-10 07:54:12,367 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-d38d8195a249477eaf5c246a883a89b1, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015650768 2023-02-10 07:54:12,368 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-c0a3ff9ba9204500912ed1612cc4ef8a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015650548 2023-02-10 07:54:12,369 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-89cad5d656774a0e9a88bad5e3824190, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015650768 2023-02-10 07:54:12,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-10 07:54:12,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure a5fd8742bb346a85e0d5f561d7ce5673, server=jenkins-hbase4.apache.org,40867,1676015645736 in 421 msec 2023-02-10 07:54:12,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-10 07:54:12,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a5fd8742bb346a85e0d5f561d7ce5673, ASSIGN in 582 msec 2023-02-10 07:54:12,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 in 1.5860 sec 2023-02-10 07:54:12,402 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] throttle.PressureAwareThroughputController(145): a5fd8742bb346a85e0d5f561d7ce5673#cf2#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:12,402 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] throttle.PressureAwareThroughputController(145): a5fd8742bb346a85e0d5f561d7ce5673#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:12,422 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/.tmp/cf1/5c5e0ca4455841dda394bcbb50ef1b3f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5c5e0ca4455841dda394bcbb50ef1b3f 2023-02-10 07:54:12,422 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/.tmp/cf2/ab9530ef0c6041ad93fa3b9b08d38ecc as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/ab9530ef0c6041ad93fa3b9b08d38ecc 2023-02-10 07:54:12,429 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a5fd8742bb346a85e0d5f561d7ce5673/cf1 of a5fd8742bb346a85e0d5f561d7ce5673 into 5c5e0ca4455841dda394bcbb50ef1b3f(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:12,429 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a5fd8742bb346a85e0d5f561d7ce5673/cf2 of a5fd8742bb346a85e0d5f561d7ce5673 into ab9530ef0c6041ad93fa3b9b08d38ecc(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:12,429 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a5fd8742bb346a85e0d5f561d7ce5673: 2023-02-10 07:54:12,430 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a5fd8742bb346a85e0d5f561d7ce5673: 2023-02-10 07:54:12,430 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673., storeName=a5fd8742bb346a85e0d5f561d7ce5673/cf1, priority=9, startTime=1676015652356; duration=0sec 2023-02-10 07:54:12,430 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673., storeName=a5fd8742bb346a85e0d5f561d7ce5673/cf2, priority=9, startTime=1676015652357; duration=0sec 2023-02-10 07:54:12,430 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:12,430 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:12,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40867] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:47068 deadline: 1676015712550, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d. is not online on jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:12,683 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-10 07:54:12,694 INFO [Listener at localhost/43795] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-10 07:54:12,696 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-10 07:54:12,726 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:12,737 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-10 07:54:12,741 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-10 07:54:12,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:12,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015652753"}]},"ts":"1676015652753"} 2023-02-10 07:54:12,756 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-10 07:54:12,758 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-10 07:54:12,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-10 07:54:12,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=817f15842cc8547f342051dd23a15f24, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a5fd8742bb346a85e0d5f561d7ce5673, UNASSIGN}] 2023-02-10 07:54:12,761 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a5fd8742bb346a85e0d5f561d7ce5673, UNASSIGN 2023-02-10 07:54:12,761 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a5fd8742bb346a85e0d5f561d7ce5673, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:12,762 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015652761"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015652761"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015652761"}]},"ts":"1676015652761"} 2023-02-10 07:54:12,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure a5fd8742bb346a85e0d5f561d7ce5673, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:12,767 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=817f15842cc8547f342051dd23a15f24, UNASSIGN 2023-02-10 07:54:12,767 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=817f15842cc8547f342051dd23a15f24, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:12,768 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015652767"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015652767"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015652767"}]},"ts":"1676015652767"} 2023-02-10 07:54:12,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 817f15842cc8547f342051dd23a15f24, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:12,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-10 07:54:12,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a5fd8742bb346a85e0d5f561d7ce5673, disabling compactions & flushes 2023-02-10 07:54:12,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. after waiting 0 ms 2023-02-10 07:54:12,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,927 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-b29cf93200c64490bc731d69520561a0, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-c0a3ff9ba9204500912ed1612cc4ef8a, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-89cad5d656774a0e9a88bad5e3824190] to archive 2023-02-10 07:54:12,929 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:12,935 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,938 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,940 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,943 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,945 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-b29cf93200c64490bc731d69520561a0 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-b29cf93200c64490bc731d69520561a0 2023-02-10 07:54:12,947 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-c0a3ff9ba9204500912ed1612cc4ef8a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-c0a3ff9ba9204500912ed1612cc4ef8a 2023-02-10 07:54:12,948 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-89cad5d656774a0e9a88bad5e3824190 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-89cad5d656774a0e9a88bad5e3824190 2023-02-10 07:54:12,968 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-22a2cc8626284278960c01b2caa18ae9, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-143b87ef94b3494ead279aac76b8f455, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-d38d8195a249477eaf5c246a883a89b1] to archive 2023-02-10 07:54:12,970 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:12,972 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,973 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,975 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,977 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,978 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-22a2cc8626284278960c01b2caa18ae9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-22a2cc8626284278960c01b2caa18ae9 2023-02-10 07:54:12,979 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-143b87ef94b3494ead279aac76b8f455 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-143b87ef94b3494ead279aac76b8f455 2023-02-10 07:54:12,981 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-d38d8195a249477eaf5c246a883a89b1 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/testSplitTableRegionUnevenDaughter=1d79b8ea1a81e86b36c3306b6c1de43d-d38d8195a249477eaf5c246a883a89b1 2023-02-10 07:54:12,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:54:12,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:12,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673. 2023-02-10 07:54:12,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a5fd8742bb346a85e0d5f561d7ce5673: Waiting for close lock at 1676015652916Running coprocessor pre-close hooks at 1676015652916Disabling compacts and flushes for region at 1676015652916Disabling writes for close at 1676015652916Writing region close event to WAL at 1676015652982 (+66 ms)Running coprocessor post-close hooks at 1676015652988 (+6 ms)Closed at 1676015652988 2023-02-10 07:54:12,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:12,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:12,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 817f15842cc8547f342051dd23a15f24, disabling compactions & flushes 2023-02-10 07:54:12,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. after waiting 0 ms 2023-02-10 07:54:12,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:12,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=a5fd8742bb346a85e0d5f561d7ce5673, regionState=CLOSED 2023-02-10 07:54:12,991 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015652990"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015652990"}]},"ts":"1676015652990"} 2023-02-10 07:54:12,991 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03-bottom] to archive 2023-02-10 07:54:12,993 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:12,995 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/6e8f8350acc147d79c8d787ef19cfbc6.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-10 07:54:12,996 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:54:12,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure a5fd8742bb346a85e0d5f561d7ce5673, server=jenkins-hbase4.apache.org,40867,1676015645736 in 230 msec 2023-02-10 07:54:12,997 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/5ec98ae4e11e411ea43598a99978c0dd.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:12,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a5fd8742bb346a85e0d5f561d7ce5673, UNASSIGN in 237 msec 2023-02-10 07:54:12,999 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,000 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/dba4b4c800ea46928eba936e6728eb03.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,002 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175-bottom] to archive 2023-02-10 07:54:13,004 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:13,005 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/4e3a8e258fc6440d844068a50c1893fe.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,007 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/9fe467b28a0d4ebb834127555eaf01d7.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,008 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/57f2feb94820493e92e74839618be3e0.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,010 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/e8122b24b60849508a9d4d75afe25175.1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:54:13,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:13,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24. 2023-02-10 07:54:13,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 817f15842cc8547f342051dd23a15f24: Waiting for close lock at 1676015652990Running coprocessor pre-close hooks at 1676015652990Disabling compacts and flushes for region at 1676015652990Disabling writes for close at 1676015652990Writing region close event to WAL at 1676015653012 (+22 ms)Running coprocessor post-close hooks at 1676015653017 (+5 ms)Closed at 1676015653018 (+1 ms) 2023-02-10 07:54:13,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:13,020 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=817f15842cc8547f342051dd23a15f24, regionState=CLOSED 2023-02-10 07:54:13,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676015653020"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015653020"}]},"ts":"1676015653020"} 2023-02-10 07:54:13,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-10 07:54:13,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 817f15842cc8547f342051dd23a15f24, server=jenkins-hbase4.apache.org,40867,1676015645736 in 253 msec 2023-02-10 07:54:13,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-10 07:54:13,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=817f15842cc8547f342051dd23a15f24, UNASSIGN in 266 msec 2023-02-10 07:54:13,029 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015653029"}]},"ts":"1676015653029"} 2023-02-10 07:54:13,030 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-10 07:54:13,032 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-10 07:54:13,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 290 msec 2023-02-10 07:54:13,061 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-10 07:54:13,062 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-10 07:54:13,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-10 07:54:13,066 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-10 07:54:13,070 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-10 07:54:13,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:13,078 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:13,080 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:13,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-10 07:54:13,088 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,088 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:13,088 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:13,093 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/recovered.edits] 2023-02-10 07:54:13,093 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/recovered.edits] 2023-02-10 07:54:13,093 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/recovered.edits] 2023-02-10 07:54:13,097 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/ea4d6822a9cb4157b90b116235070688 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf1/ea4d6822a9cb4157b90b116235070688 2023-02-10 07:54:13,098 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5c5e0ca4455841dda394bcbb50ef1b3f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf1/5c5e0ca4455841dda394bcbb50ef1b3f 2023-02-10 07:54:13,098 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/5ec98ae4e11e411ea43598a99978c0dd 2023-02-10 07:54:13,100 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/6e8f8350acc147d79c8d787ef19cfbc6 2023-02-10 07:54:13,101 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/25721165ec7d45e2b5093d1781f0ebb9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/cf2/25721165ec7d45e2b5093d1781f0ebb9 2023-02-10 07:54:13,102 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/89cad5d656774a0e9a88bad5e3824190 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/89cad5d656774a0e9a88bad5e3824190 2023-02-10 07:54:13,102 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/ab9530ef0c6041ad93fa3b9b08d38ecc to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/cf2/ab9530ef0c6041ad93fa3b9b08d38ecc 2023-02-10 07:54:13,104 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/b29cf93200c64490bc731d69520561a0 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/b29cf93200c64490bc731d69520561a0 2023-02-10 07:54:13,105 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24/recovered.edits/90.seqid 2023-02-10 07:54:13,106 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/c0a3ff9ba9204500912ed1612cc4ef8a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/c0a3ff9ba9204500912ed1612cc4ef8a 2023-02-10 07:54:13,107 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/817f15842cc8547f342051dd23a15f24 2023-02-10 07:54:13,107 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673/recovered.edits/90.seqid 2023-02-10 07:54:13,108 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:13,108 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/cd08d752c3854f7c8d730d0f5f7a6cf8 2023-02-10 07:54:13,110 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf1/dba4b4c800ea46928eba936e6728eb03 2023-02-10 07:54:13,115 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/143b87ef94b3494ead279aac76b8f455 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/143b87ef94b3494ead279aac76b8f455 2023-02-10 07:54:13,116 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/22a2cc8626284278960c01b2caa18ae9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/22a2cc8626284278960c01b2caa18ae9 2023-02-10 07:54:13,118 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/4e3a8e258fc6440d844068a50c1893fe 2023-02-10 07:54:13,120 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/57f2feb94820493e92e74839618be3e0 2023-02-10 07:54:13,121 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/9fe467b28a0d4ebb834127555eaf01d7 2023-02-10 07:54:13,123 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/d38d8195a249477eaf5c246a883a89b1 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/d38d8195a249477eaf5c246a883a89b1 2023-02-10 07:54:13,124 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/cf2/e8122b24b60849508a9d4d75afe25175 2023-02-10 07:54:13,128 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/recovered.edits/85.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d/recovered.edits/85.seqid 2023-02-10 07:54:13,128 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionUnevenDaughter/1d79b8ea1a81e86b36c3306b6c1de43d 2023-02-10 07:54:13,128 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-10 07:54:13,132 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:13,154 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-10 07:54:13,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-10 07:54:13,202 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-10 07:54:13,204 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:13,204 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-10 07:54:13,204 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015653204"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:13,204 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015653204"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:13,204 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015653204"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:13,207 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-10 07:54:13,207 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1d79b8ea1a81e86b36c3306b6c1de43d, NAME => 'testSplitTableRegionUnevenDaughter,,1676015648602.1d79b8ea1a81e86b36c3306b6c1de43d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 817f15842cc8547f342051dd23a15f24, NAME => 'testSplitTableRegionUnevenDaughter,,1676015650789.817f15842cc8547f342051dd23a15f24.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => a5fd8742bb346a85e0d5f561d7ce5673, NAME => 'testSplitTableRegionUnevenDaughter,26,1676015650789.a5fd8742bb346a85e0d5f561d7ce5673.', STARTKEY => '26', ENDKEY => ''}] 2023-02-10 07:54:13,207 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-10 07:54:13,207 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015653207"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:13,209 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-10 07:54:13,213 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:13,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 142 msec 2023-02-10 07:54:13,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-10 07:54:13,384 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-10 07:54:13,404 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=389 (was 357) Potentially hanging thread: hconnection-0x1154fe46-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38866 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1154fe46-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:48254 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38674 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38882 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38442 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:47976 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:40867-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38874 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1154fe46-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:33184 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:47944 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1154fe46-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:33068 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38638 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:48270 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:33182 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38864 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:48282 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:38608 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=859 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=326 (was 284) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3657 (was 3805) 2023-02-10 07:54:13,421 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=389, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=326, ProcessCount=168, AvailableMemoryMB=3656 2023-02-10 07:54:13,422 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:13,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:54:13,428 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:54:13,430 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:13,432 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:54:13,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:54:13,435 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,436 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3 empty. 2023-02-10 07:54:13,437 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,437 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-10 07:54:13,450 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:13,451 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1aecee45528a1e26504546f287f20ff3, NAME => 'testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:54:13,463 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:13,463 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 1aecee45528a1e26504546f287f20ff3, disabling compactions & flushes 2023-02-10 07:54:13,463 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,463 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,463 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. after waiting 0 ms 2023-02-10 07:54:13,463 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,463 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,463 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 1aecee45528a1e26504546f287f20ff3: Waiting for close lock at 1676015653463Disabling compacts and flushes for region at 1676015653463Disabling writes for close at 1676015653463Writing region close event to WAL at 1676015653463Closed at 1676015653463 2023-02-10 07:54:13,466 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:54:13,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015653467"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015653467"}]},"ts":"1676015653467"} 2023-02-10 07:54:13,469 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:54:13,470 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:54:13,470 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015653470"}]},"ts":"1676015653470"} 2023-02-10 07:54:13,472 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-10 07:54:13,475 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:13,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:13,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:13,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:13,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:13,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, ASSIGN}] 2023-02-10 07:54:13,478 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, ASSIGN 2023-02-10 07:54:13,479 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:13,629 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:13,630 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=1aecee45528a1e26504546f287f20ff3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:13,630 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015653630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015653630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015653630"}]},"ts":"1676015653630"} 2023-02-10 07:54:13,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:13,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1aecee45528a1e26504546f287f20ff3, NAME => 'testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:13,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:13,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:13,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,790 INFO [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,792 DEBUG [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1 2023-02-10 07:54:13,792 DEBUG [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1 2023-02-10 07:54:13,792 INFO [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1aecee45528a1e26504546f287f20ff3 columnFamilyName cf1 2023-02-10 07:54:13,793 INFO [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] regionserver.HStore(310): Store=1aecee45528a1e26504546f287f20ff3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:13,793 INFO [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,794 DEBUG [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2 2023-02-10 07:54:13,794 DEBUG [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2 2023-02-10 07:54:13,796 INFO [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1aecee45528a1e26504546f287f20ff3 columnFamilyName cf2 2023-02-10 07:54:13,797 INFO [StoreOpener-1aecee45528a1e26504546f287f20ff3-1] regionserver.HStore(310): Store=1aecee45528a1e26504546f287f20ff3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:13,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:13,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:13,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:13,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1aecee45528a1e26504546f287f20ff3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63751294, jitterRate=-0.050031691789627075}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:13,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1aecee45528a1e26504546f287f20ff3: Running coprocessor pre-open hook at 1676015653789Writing region info on filesystem at 1676015653789Initializing all the Stores at 1676015653790 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015653790Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015653790Cleaning up temporary data from old regions at 1676015653799 (+9 ms)Cleaning up detritus from prior splits at 1676015653800 (+1 ms)Running coprocessor post-open hooks at 1676015653805 (+5 ms)Region opened successfully at 1676015653805 2023-02-10 07:54:13,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3., pid=27, masterSystemTime=1676015653785 2023-02-10 07:54:13,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,808 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=1aecee45528a1e26504546f287f20ff3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:13,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015653808"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015653808"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015653808"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015653808"}]},"ts":"1676015653808"} 2023-02-10 07:54:13,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-10 07:54:13,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 in 178 msec 2023-02-10 07:54:13,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-10 07:54:13,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, ASSIGN in 337 msec 2023-02-10 07:54:13,816 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:54:13,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015653816"}]},"ts":"1676015653816"} 2023-02-10 07:54:13,818 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-10 07:54:13,821 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:54:13,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 392 msec 2023-02-10 07:54:13,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:13,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-10 07:54:13,944 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-10 07:54:13,944 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:13,945 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-10 07:54:13,945 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-10 07:54:13,945 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:13,946 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:13,947 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:13,947 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:13,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:13,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:13,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:13,947 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:13,947 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:13,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:13,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:13,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:13,947 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:13,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:13,948 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:13,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:13,949 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:13,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:13,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:13,949 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:13,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:13,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:13,951 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:13,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:13,951 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:13,951 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:13,952 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:13,953 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:13,953 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:13,953 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:13,953 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:13,953 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:13,953 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. started... 2023-02-10 07:54:13,953 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:13,954 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1aecee45528a1e26504546f287f20ff3 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:54:13,954 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:13,955 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:13,956 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:13,967 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/36fb54c46f4046ac96e4dedf4a858822 2023-02-10 07:54:13,985 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/b6806e3f59ae47509a024dc28708755c 2023-02-10 07:54:13,992 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/36fb54c46f4046ac96e4dedf4a858822 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822 2023-02-10 07:54:13,997 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:13,998 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/b6806e3f59ae47509a024dc28708755c as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c 2023-02-10 07:54:14,004 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:14,005 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 51ms, sequenceid=6, compaction requested=false 2023-02-10 07:54:14,005 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-10 07:54:14,006 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1aecee45528a1e26504546f287f20ff3: 2023-02-10 07:54:14,006 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,006 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:14,006 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:14,006 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,007 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,007 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,010 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,010 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,010 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,010 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-10 07:54:14,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cae0c80[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:14,012 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-10 07:54:14,012 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,013 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,013 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,013 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,014 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,014 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:14,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,014 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:14,015 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,015 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,016 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,016 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,017 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,017 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,017 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,017 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,017 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,018 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:14,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-10 07:54:14,021 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-10 07:54:14,022 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:14,022 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:14,022 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-10 07:54:14,022 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:14,024 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,024 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,025 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,026 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,026 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,028 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,028 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,028 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,031 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-10 07:54:14,031 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:14,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,031 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,031 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,032 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:14,032 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,033 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:14,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,033 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,033 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,033 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,033 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,033 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,133 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:14,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:14,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:14,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:14,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-10 07:54:14,153 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-10 07:54:14,153 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,154 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-10 07:54:14,154 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-10 07:54:14,154 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,154 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,156 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,156 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,156 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:14,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,157 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,157 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,158 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,158 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,158 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,158 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,158 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,159 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,159 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,159 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,159 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,159 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,159 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,159 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,159 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,160 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,160 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,160 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. started... 2023-02-10 07:54:14,161 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1aecee45528a1e26504546f287f20ff3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:14,162 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,162 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,162 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,162 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,162 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,162 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,172 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/d210420e123040aa81ab1f37ae7962a9 2023-02-10 07:54:14,188 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/13d521e2dc4946d39ed732b00b65bb55 2023-02-10 07:54:14,193 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/d210420e123040aa81ab1f37ae7962a9 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9 2023-02-10 07:54:14,199 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:14,200 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/13d521e2dc4946d39ed732b00b65bb55 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55 2023-02-10 07:54:14,204 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:14,205 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 44ms, sequenceid=19, compaction requested=false 2023-02-10 07:54:14,206 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1aecee45528a1e26504546f287f20ff3: 2023-02-10 07:54:14,206 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,206 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:14,206 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:14,206 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,206 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,206 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,208 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,208 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,208 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,208 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-10 07:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4530c1e5[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:14,210 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-10 07:54:14,210 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,211 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,211 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,211 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,212 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,212 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,212 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:14,213 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,213 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,213 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,213 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,218 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,218 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,218 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,219 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,219 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,219 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:14,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-10 07:54:14,223 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-10 07:54:14,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:14,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:14,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-10 07:54:14,223 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:14,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,225 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,226 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,227 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,228 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,229 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,229 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-10 07:54:14,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,233 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,233 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,235 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,235 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:14,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,234 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,235 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:14,235 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,236 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,236 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,236 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,236 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,335 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:14,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:14,354 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:14,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:14,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-10 07:54:14,355 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-10 07:54:14,355 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,356 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-10 07:54:14,356 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-10 07:54:14,357 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,357 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,361 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,361 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,361 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:14,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,363 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,363 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,363 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,363 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,363 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,363 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,363 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,363 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,363 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,363 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,364 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,365 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,365 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:14,365 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:14,365 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,365 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. started... 2023-02-10 07:54:14,365 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1aecee45528a1e26504546f287f20ff3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:14,366 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,366 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,367 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,367 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,367 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,367 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,377 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/ad0f629d01294b8e8190834c9c4dff51 2023-02-10 07:54:14,394 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/cc18ac59a1d24701ac17e97181b74f80 2023-02-10 07:54:14,400 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/ad0f629d01294b8e8190834c9c4dff51 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51 2023-02-10 07:54:14,406 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:14,406 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/cc18ac59a1d24701ac17e97181b74f80 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80 2023-02-10 07:54:14,411 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:14,412 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 47ms, sequenceid=32, compaction requested=true 2023-02-10 07:54:14,412 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1aecee45528a1e26504546f287f20ff3: 2023-02-10 07:54:14,412 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,412 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:14,412 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:14,412 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,412 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,412 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,414 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,414 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,414 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,414 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-10 07:54:14,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@149e56c4[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:14,416 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-10 07:54:14,416 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,417 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,417 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,417 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,418 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,418 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:14,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,418 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:14,421 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,421 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,421 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,421 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,421 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,421 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,422 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,423 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,423 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,423 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:14,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-10 07:54:14,426 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-10 07:54:14,426 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:14,426 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:14,426 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-10 07:54:14,426 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:14,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,428 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,429 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,430 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,432 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,432 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,433 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,436 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-10 07:54:14,437 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,436 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,437 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,437 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:14,436 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,438 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,438 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,438 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,438 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:14,438 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:14,539 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:14,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:14,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:14,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:14,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-10 07:54:14,560 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-10 07:54:14,560 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,561 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-10 07:54:14,561 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-10 07:54:14,561 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,562 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,563 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,563 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,563 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,563 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,564 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,564 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,565 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,565 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,565 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,567 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,568 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,568 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,568 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,568 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,568 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,571 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,571 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:14,571 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,571 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:14,571 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,571 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. started... 2023-02-10 07:54:14,571 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,571 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1aecee45528a1e26504546f287f20ff3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:14,575 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,577 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,578 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,582 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/35aa46cd8b524270ab1723e7bdf8ca73 2023-02-10 07:54:14,598 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/78cf0a694e26483695ddc4be9e5e44ec 2023-02-10 07:54:14,606 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/35aa46cd8b524270ab1723e7bdf8ca73 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73 2023-02-10 07:54:14,611 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:14,612 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/78cf0a694e26483695ddc4be9e5e44ec as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec 2023-02-10 07:54:14,616 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:14,617 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 46ms, sequenceid=45, compaction requested=true 2023-02-10 07:54:14,617 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1aecee45528a1e26504546f287f20ff3: 2023-02-10 07:54:14,617 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,618 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:14,618 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:14,618 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,618 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,618 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,620 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,620 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,620 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,621 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-10 07:54:14,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43de81dc[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:14,622 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-10 07:54:14,622 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,623 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,624 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:14,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:14,624 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:14,625 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,625 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,625 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,626 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,626 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,626 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,626 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,626 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,627 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,627 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:14,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-10 07:54:14,632 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-10 07:54:14,632 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:14,632 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:14,632 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-10 07:54:14,632 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:14,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,635 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,635 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,635 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,635 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,636 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,637 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,638 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,638 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-10 07:54:14,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:14,641 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,641 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:14,642 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:14,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,676 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-10 07:54:14,677 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-10 07:54:14,677 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:14,677 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-10 07:54:14,742 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:14,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:14,762 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:14,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:14,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-10 07:54:14,764 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-10 07:54:14,764 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,764 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-10 07:54:14,764 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-10 07:54:14,764 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,764 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,765 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,765 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,765 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,765 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,766 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,766 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,767 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,767 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,767 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:14,768 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:14,768 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,768 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. started... 2023-02-10 07:54:14,768 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1aecee45528a1e26504546f287f20ff3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:14,769 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,771 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,771 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,771 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,771 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,771 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,786 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/b580a4cae2294428bb31a97ef4fe01e6 2023-02-10 07:54:14,801 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/43aa087ffa7b4a2689f2a91e45e64189 2023-02-10 07:54:14,808 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/b580a4cae2294428bb31a97ef4fe01e6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6 2023-02-10 07:54:14,814 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:14,815 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/43aa087ffa7b4a2689f2a91e45e64189 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189 2023-02-10 07:54:14,825 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:14,826 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 58ms, sequenceid=58, compaction requested=true 2023-02-10 07:54:14,826 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1aecee45528a1e26504546f287f20ff3: 2023-02-10 07:54:14,826 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,826 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:14,826 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:14,826 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,826 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,826 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,828 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,828 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,828 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,828 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-10 07:54:14,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fd84f2b[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:14,831 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-10 07:54:14,831 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,832 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,832 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,832 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,833 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,833 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:14,833 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:14,834 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,834 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,835 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,837 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,837 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,837 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:14,837 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,837 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,837 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,837 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:14,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:14,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-10 07:54:14,840 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-10 07:54:14,840 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:14,840 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:14,840 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-10 07:54:14,840 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:14,841 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,841 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,841 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,841 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,841 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:14,842 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,843 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:14,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,844 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:14,845 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:14,845 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:14,852 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-10 07:54:14,852 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:14,852 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:14,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,852 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:14,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:14,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:14,853 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,854 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,854 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,953 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:14,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:14,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:14,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:14,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-10 07:54:14,974 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-10 07:54:14,974 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,974 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-10 07:54:14,974 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-10 07:54:14,974 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,974 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,977 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,977 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,977 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,977 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:14,977 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,977 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:14,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-10 07:54:14,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:14,980 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-10 07:54:14,980 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:14,980 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:14,980 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:14,980 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:14,980 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. started... 2023-02-10 07:54:14,980 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1aecee45528a1e26504546f287f20ff3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:14,981 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,981 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,981 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,982 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,982 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:14,982 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:14,992 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/3ac7562f8e8f4d1f93b784e394c79a55 2023-02-10 07:54:15,007 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/8d058acda94641bcaa5373399d28b5ff 2023-02-10 07:54:15,012 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/3ac7562f8e8f4d1f93b784e394c79a55 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55 2023-02-10 07:54:15,017 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:15,018 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/8d058acda94641bcaa5373399d28b5ff as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff 2023-02-10 07:54:15,022 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:15,023 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 43ms, sequenceid=71, compaction requested=true 2023-02-10 07:54:15,023 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1aecee45528a1e26504546f287f20ff3: 2023-02-10 07:54:15,024 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:15,024 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:15,024 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:15,024 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,024 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-10 07:54:15,024 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-10 07:54:15,025 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,025 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:15,026 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,026 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:15,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-10 07:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ca488dc[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:15,028 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-10 07:54:15,028 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,030 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:15,030 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:15,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,031 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:15,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-10 07:54:15,031 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:15,032 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:15,032 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,032 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:15,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:15,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:15,032 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:15,033 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:15,033 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:15,033 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:15,034 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-10 07:54:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:15,034 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:15,034 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-10 07:54:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:15,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:15,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-10 07:54:15,037 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-10 07:54:15,037 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:15,037 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:15,037 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-10 07:54:15,037 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:15,039 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:15,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:15,039 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:15,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,039 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:15,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:15,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:15,040 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:15,041 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:15,041 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:15,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:15,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,042 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,043 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:15,043 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-10 07:54:15,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,045 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,045 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:15,045 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:15,045 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:15,045 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,046 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,046 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-10 07:54:15,046 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:15,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:15,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:15,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:15,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:15,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:15,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:15,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:15,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:15,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-10 07:54:15,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:15,046 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:15,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:15,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:15,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:15,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:15,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-10 07:54:15,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-10 07:54:15,147 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:15,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:15,410 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-10 07:54:15,411 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:15,412 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,413 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,413 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,413 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,414 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:15,414 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:15,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:15,663 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,664 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:15,664 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:15,664 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:15,666 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-10 07:54:15,666 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:15,667 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:15,667 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:15,667 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:15,667 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:15,670 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-10 07:54:15,671 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:15,672 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:15,672 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:15,672 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:15,673 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:15,673 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:15,673 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:15,674 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:15,674 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:15,675 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,675 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:15,675 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,675 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,675 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,675 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-10 07:54:15,676 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:15,677 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:15,681 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015685681 2023-02-10 07:54:15,682 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,682 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:15,687 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:15,687 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:15,687 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:15,687 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015715687 2023-02-10 07:54:15,687 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015775687 2023-02-10 07:54:15,687 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-10 07:54:15,688 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:15,690 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:15,690 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:15,690 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:15,938 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,938 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:15,938 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:15,938 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:15,939 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:15,939 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:15,939 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:15,939 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:15,940 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:15,940 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:15,942 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-10 07:54:15,943 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:15,943 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:15,944 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:15,944 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:15,944 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:15,945 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:15,945 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:15,945 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:15,945 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:15,946 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,946 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:15,946 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,946 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,946 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,946 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:54:15,947 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:15,948 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:15,948 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015685948 2023-02-10 07:54:15,948 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:15,949 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,952 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:15,952 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:15,952 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:15,953 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015715953 2023-02-10 07:54:15,953 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015775953 2023-02-10 07:54:15,953 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:15,953 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:15,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN}] 2023-02-10 07:54:15,955 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:15,955 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:15,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:15,955 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:16,203 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,203 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:16,203 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:16,203 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:16,204 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:16,204 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:16,204 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:16,205 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:16,205 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:16,205 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:16,207 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-10 07:54:16,207 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:16,207 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:16,208 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:16,208 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:16,208 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,209 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:16,209 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,209 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,210 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:16,210 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,210 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:16,210 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,210 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,210 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,211 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:16,211 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:16,212 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:16,215 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015686215 2023-02-10 07:54:16,216 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:16,216 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,219 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:16,219 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:16,219 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:16,219 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015716219 2023-02-10 07:54:16,219 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015776219 2023-02-10 07:54:16,219 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:16,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN}] 2023-02-10 07:54:16,220 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:16,222 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1aecee45528a1e26504546f287f20ff3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:16,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015656223"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015656223"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015656223"}]},"ts":"1676015656223"} 2023-02-10 07:54:16,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:16,224 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:16,224 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:16,224 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:16,470 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:16,470 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:16,471 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:16,473 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:16,473 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-10 07:54:16,473 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:16,474 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:16,474 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:16,474 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:16,475 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,475 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:16,475 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,475 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,476 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:16,477 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,477 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,477 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:16,477 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,477 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,477 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,477 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:16,477 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,477 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:54:16,478 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:16,478 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:16,479 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3 to restore RIT 2023-02-10 07:54:16,479 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015686479 2023-02-10 07:54:16,480 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:16,480 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,482 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:16,482 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:16,482 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:16,483 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015716483 2023-02-10 07:54:16,483 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015776483 2023-02-10 07:54:16,483 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:54:16,483 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1aecee45528a1e26504546f287f20ff3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:16,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015656483"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015656483"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015656483"}]},"ts":"1676015656483"} 2023-02-10 07:54:16,485 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:16,485 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:16,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:16,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1aecee45528a1e26504546f287f20ff3, disabling compactions & flushes 2023-02-10 07:54:16,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:16,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:16,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. after waiting 0 ms 2023-02-10 07:54:16,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:16,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1aecee45528a1e26504546f287f20ff3 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-10 07:54:16,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 2023-02-10 07:54:16,666 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/adfb8fa1b6634eca980c99f7a91ea55a 2023-02-10 07:54:16,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 2023-02-10 07:54:16,676 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:54:16,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/.tmp/cf2/adfb8fa1b6634eca980c99f7a91ea55a as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a 2023-02-10 07:54:16,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:54:16,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 45ms, sequenceid=82, compaction requested=true 2023-02-10 07:54:16,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-10 07:54:16,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:16,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. 2023-02-10 07:54:16,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1aecee45528a1e26504546f287f20ff3: Waiting for close lock at 1676015656638Running coprocessor pre-close hooks at 1676015656638Disabling compacts and flushes for region at 1676015656638Disabling writes for close at 1676015656638Obtaining lock to block concurrent updates at 1676015656638Preparing flush snapshotting stores in 1aecee45528a1e26504546f287f20ff3 at 1676015656638Finished memstore snapshotting testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676015656638Flushing stores of testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. at 1676015656639 (+1 ms)Flushing 1aecee45528a1e26504546f287f20ff3/cf1: creating writer at 1676015656640 (+1 ms)Flushing 1aecee45528a1e26504546f287f20ff3/cf1: appending metadata at 1676015656642 (+2 ms)Flushing 1aecee45528a1e26504546f287f20ff3/cf1: closing flushed file at 1676015656642Flushing 1aecee45528a1e26504546f287f20ff3/cf2: creating writer at 1676015656655 (+13 ms)Flushing 1aecee45528a1e26504546f287f20ff3/cf2: appending metadata at 1676015656657 (+2 ms)Flushing 1aecee45528a1e26504546f287f20ff3/cf2: closing flushed file at 1676015656657Flushing 1aecee45528a1e26504546f287f20ff3/cf1: reopening flushed file at 1676015656672 (+15 ms)Flushing 1aecee45528a1e26504546f287f20ff3/cf2: reopening flushed file at 1676015656678 (+6 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 1aecee45528a1e26504546f287f20ff3 in 45ms, sequenceid=82, compaction requested=true at 1676015656683 (+5 ms)Writing region close event to WAL at 1676015656699 (+16 ms)Running coprocessor post-close hooks at 1676015656703 (+4 ms)Closed at 1676015656703 2023-02-10 07:54:16,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:16,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED 2023-02-10 07:54:16,706 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015656706"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015656706"}]},"ts":"1676015656706"} 2023-02-10 07:54:16,707 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:16,707 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:16,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:16,708 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:16,733 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,733 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:16,733 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:16,733 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:16,734 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:16,734 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:16,734 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:16,734 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:16,734 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:16,735 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:16,736 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:16,736 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-10 07:54:16,736 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-10 07:54:16,737 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:16,737 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:16,737 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:16,737 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:16,738 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,738 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:16,738 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,739 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:16,739 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:16,740 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,740 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,740 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:16,740 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:16,740 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,740 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,740 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,740 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:16,740 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,741 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:16,741 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:16,741 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:16,742 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:16,742 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3 to restore RIT 2023-02-10 07:54:16,742 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015686742 2023-02-10 07:54:16,743 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:16,743 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:16,745 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:16,746 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:16,746 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:16,746 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015716746 2023-02-10 07:54:16,746 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015776746 2023-02-10 07:54:16,746 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:16,746 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:16,747 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015656746"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015656746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015656746"}]},"ts":"1676015656746"} 2023-02-10 07:54:16,748 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:16,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-10 07:54:16,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 in 263 msec 2023-02-10 07:54:16,752 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:16,752 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:16,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:16,752 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:16,996 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:16,997 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:16,997 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:16,997 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:16,998 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:16,998 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:16,998 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:16,998 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:16,998 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:16,999 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:17,000 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:17,001 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-10 07:54:17,001 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:17,001 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:17,002 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:17,002 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:17,003 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:17,003 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:17,004 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:17,004 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:17,004 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:17,005 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:17,006 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,006 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:17,006 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:17,006 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:17,006 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,006 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,007 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,007 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:17,007 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:17,007 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-10 07:54:17,007 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:17,008 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:17,008 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3 to restore RIT 2023-02-10 07:54:17,008 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015687008 2023-02-10 07:54:17,009 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:17,009 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:17,012 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:17,012 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:17,012 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:17,012 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015717012 2023-02-10 07:54:17,012 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015777012 2023-02-10 07:54:17,012 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:17,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:17,014 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:17,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-10 07:54:17,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN in 794 msec 2023-02-10 07:54:17,021 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:17,021 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:17,021 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:17,263 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,263 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:17,263 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:17,263 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:17,264 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:17,264 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:17,264 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:17,264 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:17,264 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:17,265 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:17,266 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:17,266 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:17,266 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:17,266 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:17,267 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:17,267 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:17,267 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:17,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:17,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:17,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:17,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:17,269 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:17,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:17,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:17,270 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:17,270 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:17,270 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:17,270 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:17,271 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:17,271 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:17,272 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015687272 2023-02-10 07:54:17,273 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,273 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:17,276 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:17,276 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:17,276 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:17,276 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015717276 2023-02-10 07:54:17,276 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015777276 2023-02-10 07:54:17,276 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:17,276 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,282 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:17,291 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=1aecee45528a1e26504546f287f20ff3, threads=14 2023-02-10 07:54:17,292 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,292 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,293 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,296 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,296 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,299 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,299 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,417 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,422 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,444 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,449 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,453 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,456 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,458 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,461 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,468 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,468 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,473 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,474 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,478 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,821 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:17,822 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 1aecee45528a1e26504546f287f20ff3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:54:17,870 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:17,870 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:17,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:17,870 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:18,027 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,027 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:18,027 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:18,027 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:18,028 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:18,028 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:18,028 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:18,028 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:18,028 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:18,029 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:18,030 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-10 07:54:18,030 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:18,030 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:18,031 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:18,031 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:18,032 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:18,032 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:18,032 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,033 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:18,033 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,034 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,034 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:18,035 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,035 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:18,035 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:18,035 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:18,036 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,036 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,036 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,036 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:18,036 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:18,036 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:54:18,037 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:18,037 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:18,038 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015688038 2023-02-10 07:54:18,039 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:18,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,042 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:18,042 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:18,042 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:18,042 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015718042 2023-02-10 07:54:18,042 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015778042 2023-02-10 07:54:18,042 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:18,053 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=1aecee45528a1e26504546f287f20ff3, threads=14 2023-02-10 07:54:18,054 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,054 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,054 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,054 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,055 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,054 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,055 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,055 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,055 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,055 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,062 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,062 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,063 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,063 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,190 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,193 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,244 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,244 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,268 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,285 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,288 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,296 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,298 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,298 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,300 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,300 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,304 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,307 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9 for region: 1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:18,307 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 1aecee45528a1e26504546f287f20ff3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:54:18,357 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:18,363 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:54:18,366 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:54:18,366 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:18,366 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:18,366 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:18,490 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:54:18,558 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,559 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:18,559 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:18,559 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:18,559 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:18,562 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:18,562 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:18,562 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:18,562 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:18,563 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:18,569 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-10 07:54:18,569 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:18,569 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:18,569 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:18,570 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:18,570 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:18,570 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:18,570 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,571 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:18,571 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,571 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,571 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:18,572 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,572 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:18,572 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:18,572 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:18,572 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,572 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,572 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,572 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:18,572 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:18,573 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-10 07:54:18,573 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:18,574 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:18,578 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015688578 2023-02-10 07:54:18,579 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:18,579 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,582 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:18,582 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:18,582 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:18,582 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015718582 2023-02-10 07:54:18,582 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015778582 2023-02-10 07:54:18,582 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:54:18,585 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:18,588 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:18,588 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:18,588 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:18,833 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:18,833 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:18,833 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:18,833 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:18,834 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:18,834 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:18,834 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:18,834 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:18,834 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:18,835 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:18,836 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-10 07:54:18,836 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:18,836 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:18,836 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:18,837 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:18,837 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:18,837 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:18,837 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,838 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:18,838 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,838 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:18,838 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:18,839 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,839 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN 2023-02-10 07:54:18,839 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:18,839 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:18,839 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,839 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,839 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,839 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=1aecee45528a1e26504546f287f20ff3, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:18,839 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 1aecee45528a1e26504546f287f20ff3, server=jenkins-hbase4.apache.org,40867,1676015645736 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:18,839 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-10 07:54:18,840 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:18,841 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:18,841 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015688841 2023-02-10 07:54:18,842 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:18,842 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:18,844 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:18,845 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=1aecee45528a1e26504546f287f20ff3, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:18,845 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:18,845 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015718845 2023-02-10 07:54:18,845 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015778845 2023-02-10 07:54:18,845 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:54:18,848 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015658848"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676015658848"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676015658848"}]},"ts":"1676015658848"} 2023-02-10 07:54:18,848 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015658848"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015658848"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015658848"}]},"ts":"1676015658848"} 2023-02-10 07:54:18,848 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015658848"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015658848"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015658848"}]},"ts":"1676015658848"} 2023-02-10 07:54:18,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3817a509de9c51aaf4f3e11d1b870d42, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=39a95a9bfb795ab2b5aa9c3c4cbfb443, ASSIGN}] 2023-02-10 07:54:18,855 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=39a95a9bfb795ab2b5aa9c3c4cbfb443, ASSIGN 2023-02-10 07:54:18,855 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=39a95a9bfb795ab2b5aa9c3c4cbfb443, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:18,856 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3817a509de9c51aaf4f3e11d1b870d42, ASSIGN 2023-02-10 07:54:18,857 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3817a509de9c51aaf4f3e11d1b870d42, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:19,006 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-10 07:54:19,007 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=3817a509de9c51aaf4f3e11d1b870d42, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:19,008 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659007"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015659007"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015659007"}]},"ts":"1676015659007"} 2023-02-10 07:54:19,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 3817a509de9c51aaf4f3e11d1b870d42, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:19,013 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=39a95a9bfb795ab2b5aa9c3c4cbfb443, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:19,013 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659013"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015659013"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015659013"}]},"ts":"1676015659013"} 2023-02-10 07:54:19,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 39a95a9bfb795ab2b5aa9c3c4cbfb443, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:19,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 39a95a9bfb795ab2b5aa9c3c4cbfb443, NAME => 'testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.', STARTKEY => '41', ENDKEY => ''} 2023-02-10 07:54:19,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:19,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:19,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,167 INFO [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,168 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1 2023-02-10 07:54:19,169 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1 2023-02-10 07:54:19,169 INFO [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39a95a9bfb795ab2b5aa9c3c4cbfb443 columnFamilyName cf1 2023-02-10 07:54:19,196 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73-top 2023-02-10 07:54:19,202 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822-top 2023-02-10 07:54:19,207 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55-top 2023-02-10 07:54:19,214 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1-top 2023-02-10 07:54:19,219 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51-top 2023-02-10 07:54:19,224 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6-top 2023-02-10 07:54:19,229 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9-top 2023-02-10 07:54:19,229 INFO [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(310): Store=39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:19,230 INFO [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,231 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2 2023-02-10 07:54:19,231 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2 2023-02-10 07:54:19,231 INFO [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39a95a9bfb795ab2b5aa9c3c4cbfb443 columnFamilyName cf2 2023-02-10 07:54:19,249 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55-top 2023-02-10 07:54:19,254 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189-top 2023-02-10 07:54:19,260 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec-top 2023-02-10 07:54:19,264 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff-top 2023-02-10 07:54:19,270 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a-top 2023-02-10 07:54:19,278 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c-top 2023-02-10 07:54:19,284 DEBUG [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80-top 2023-02-10 07:54:19,285 INFO [StoreOpener-39a95a9bfb795ab2b5aa9c3c4cbfb443-1] regionserver.HStore(310): Store=39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:19,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:19,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 39a95a9bfb795ab2b5aa9c3c4cbfb443; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65427681, jitterRate=-0.025051578879356384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:19,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 39a95a9bfb795ab2b5aa9c3c4cbfb443: Running coprocessor pre-open hook at 1676015659166Writing region info on filesystem at 1676015659166Initializing all the Stores at 1676015659167 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015659167Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015659167Cleaning up temporary data from old regions at 1676015659292 (+125 ms)Cleaning up detritus from prior splits at 1676015659293 (+1 ms)Running coprocessor post-open hooks at 1676015659297 (+4 ms)Region opened successfully at 1676015659297 2023-02-10 07:54:19,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443., pid=34, masterSystemTime=1676015659162 2023-02-10 07:54:19,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:19,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:19,300 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:19,300 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:19,300 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,300 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1912): 39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2 is initiating minor compaction (all files) 2023-02-10 07:54:19,300 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,300 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2 in testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,300 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1912): 39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1 is initiating minor compaction (all files) 2023-02-10 07:54:19,300 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1 in testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,301 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1-top] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/.tmp, totalSize=34.9 K 2023-02-10 07:54:19,301 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a-top] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/.tmp, totalSize=34.9 K 2023-02-10 07:54:19,302 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015653934 2023-02-10 07:54:19,302 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015653934 2023-02-10 07:54:19,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3817a509de9c51aaf4f3e11d1b870d42, NAME => 'testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.', STARTKEY => '', ENDKEY => '41'} 2023-02-10 07:54:19,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:19,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:19,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,308 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=39a95a9bfb795ab2b5aa9c3c4cbfb443, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:19,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659308"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015659308"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015659308"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015659308"}]},"ts":"1676015659308"} 2023-02-10 07:54:19,309 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015654135 2023-02-10 07:54:19,309 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015654135 2023-02-10 07:54:19,309 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015654338 2023-02-10 07:54:19,309 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015654338 2023-02-10 07:54:19,310 INFO [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,311 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015654541 2023-02-10 07:54:19,312 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015654541 2023-02-10 07:54:19,314 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1 2023-02-10 07:54:19,314 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676015654745 2023-02-10 07:54:19,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-10 07:54:19,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 39a95a9bfb795ab2b5aa9c3c4cbfb443, server=jenkins-hbase4.apache.org,40867,1676015645736 in 297 msec 2023-02-10 07:54:19,314 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676015654745 2023-02-10 07:54:19,314 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1 2023-02-10 07:54:19,315 INFO [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3817a509de9c51aaf4f3e11d1b870d42 columnFamilyName cf1 2023-02-10 07:54:19,315 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676015654956 2023-02-10 07:54:19,315 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676015654956 2023-02-10 07:54:19,316 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015655149 2023-02-10 07:54:19,316 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015655149 2023-02-10 07:54:19,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=39a95a9bfb795ab2b5aa9c3c4cbfb443, ASSIGN in 462 msec 2023-02-10 07:54:19,342 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] throttle.PressureAwareThroughputController(145): 39a95a9bfb795ab2b5aa9c3c4cbfb443#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:19,342 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] throttle.PressureAwareThroughputController(145): 39a95a9bfb795ab2b5aa9c3c4cbfb443#cf2#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:19,345 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73-bottom 2023-02-10 07:54:19,349 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822-bottom 2023-02-10 07:54:19,359 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55-bottom 2023-02-10 07:54:19,364 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1-bottom 2023-02-10 07:54:19,379 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51-bottom 2023-02-10 07:54:19,384 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6-bottom 2023-02-10 07:54:19,393 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9-bottom 2023-02-10 07:54:19,404 INFO [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(310): Store=3817a509de9c51aaf4f3e11d1b870d42/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:19,404 INFO [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,405 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2 2023-02-10 07:54:19,406 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2 2023-02-10 07:54:19,406 INFO [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3817a509de9c51aaf4f3e11d1b870d42 columnFamilyName cf2 2023-02-10 07:54:19,417 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/.tmp/cf1/9bf91a0e67514fff9a16cf669539c04d as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/9bf91a0e67514fff9a16cf669539c04d 2023-02-10 07:54:19,428 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/.tmp/cf2/2dd35571eb5c4d23b51886b77088f12a as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/2dd35571eb5c4d23b51886b77088f12a 2023-02-10 07:54:19,431 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1 of 39a95a9bfb795ab2b5aa9c3c4cbfb443 into 9bf91a0e67514fff9a16cf669539c04d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:19,431 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 39a95a9bfb795ab2b5aa9c3c4cbfb443: 2023-02-10 07:54:19,431 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443., storeName=39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1, priority=9, startTime=1676015659299; duration=0sec 2023-02-10 07:54:19,432 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:19,438 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55-bottom 2023-02-10 07:54:19,442 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2 of 39a95a9bfb795ab2b5aa9c3c4cbfb443 into 2dd35571eb5c4d23b51886b77088f12a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:19,442 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 39a95a9bfb795ab2b5aa9c3c4cbfb443: 2023-02-10 07:54:19,442 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443., storeName=39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2, priority=9, startTime=1676015659299; duration=0sec 2023-02-10 07:54:19,442 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:19,443 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189-bottom 2023-02-10 07:54:19,448 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec-bottom 2023-02-10 07:54:19,453 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff-bottom 2023-02-10 07:54:19,459 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a-bottom 2023-02-10 07:54:19,464 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c-bottom 2023-02-10 07:54:19,470 DEBUG [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80-bottom 2023-02-10 07:54:19,471 INFO [StoreOpener-3817a509de9c51aaf4f3e11d1b870d42-1] regionserver.HStore(310): Store=3817a509de9c51aaf4f3e11d1b870d42/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:19,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:19,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3817a509de9c51aaf4f3e11d1b870d42; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62382404, jitterRate=-0.0704297423362732}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:19,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3817a509de9c51aaf4f3e11d1b870d42: Running coprocessor pre-open hook at 1676015659304Writing region info on filesystem at 1676015659304Initializing all the Stores at 1676015659309 (+5 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015659309Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015659310 (+1 ms)Cleaning up temporary data from old regions at 1676015659474 (+164 ms)Cleaning up detritus from prior splits at 1676015659474Running coprocessor post-open hooks at 1676015659477 (+3 ms)Region opened successfully at 1676015659477 2023-02-10 07:54:19,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42., pid=33, masterSystemTime=1676015659162 2023-02-10 07:54:19,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:19,479 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:19,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:19,479 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:19,479 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,479 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1912): 3817a509de9c51aaf4f3e11d1b870d42/cf1 is initiating minor compaction (all files) 2023-02-10 07:54:19,479 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,480 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3817a509de9c51aaf4f3e11d1b870d42/cf1 in testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,480 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1912): 3817a509de9c51aaf4f3e11d1b870d42/cf2 is initiating minor compaction (all files) 2023-02-10 07:54:19,480 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3817a509de9c51aaf4f3e11d1b870d42/cf2 in testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,480 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/.tmp, totalSize=34.9 K 2023-02-10 07:54:19,480 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/.tmp, totalSize=34.9 K 2023-02-10 07:54:19,481 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015653934 2023-02-10 07:54:19,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,482 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015653934 2023-02-10 07:54:19,482 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015654135 2023-02-10 07:54:19,482 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015654135 2023-02-10 07:54:19,482 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=3817a509de9c51aaf4f3e11d1b870d42, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:19,482 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015654338 2023-02-10 07:54:19,483 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659482"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015659482"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015659482"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015659482"}]},"ts":"1676015659482"} 2023-02-10 07:54:19,483 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015654338 2023-02-10 07:54:19,484 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015654541 2023-02-10 07:54:19,486 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015654541 2023-02-10 07:54:19,486 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015654745 2023-02-10 07:54:19,486 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015654745 2023-02-10 07:54:19,486 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015654956 2023-02-10 07:54:19,487 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015654956 2023-02-10 07:54:19,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-10 07:54:19,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 3817a509de9c51aaf4f3e11d1b870d42, server=jenkins-hbase4.apache.org,40867,1676015645736 in 475 msec 2023-02-10 07:54:19,487 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015655149 2023-02-10 07:54:19,487 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015655149 2023-02-10 07:54:19,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-10 07:54:19,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3817a509de9c51aaf4f3e11d1b870d42, ASSIGN in 635 msec 2023-02-10 07:54:19,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 in 4.0810 sec 2023-02-10 07:54:19,507 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] throttle.PressureAwareThroughputController(145): 3817a509de9c51aaf4f3e11d1b870d42#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:19,510 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3817a509de9c51aaf4f3e11d1b870d42#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:19,527 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/.tmp/cf1/a6faf759c77b4b58925ed4fca5ca0f67 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/a6faf759c77b4b58925ed4fca5ca0f67 2023-02-10 07:54:19,527 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/.tmp/cf2/b953cd6fcd41439ca4a4f77fe94c446e as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b953cd6fcd41439ca4a4f77fe94c446e 2023-02-10 07:54:19,532 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3817a509de9c51aaf4f3e11d1b870d42/cf1 of 3817a509de9c51aaf4f3e11d1b870d42 into a6faf759c77b4b58925ed4fca5ca0f67(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:19,533 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3817a509de9c51aaf4f3e11d1b870d42: 2023-02-10 07:54:19,533 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42., storeName=3817a509de9c51aaf4f3e11d1b870d42/cf1, priority=9, startTime=1676015659478; duration=0sec 2023-02-10 07:54:19,533 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:19,533 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3817a509de9c51aaf4f3e11d1b870d42/cf2 of 3817a509de9c51aaf4f3e11d1b870d42 into b953cd6fcd41439ca4a4f77fe94c446e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:19,533 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3817a509de9c51aaf4f3e11d1b870d42: 2023-02-10 07:54:19,533 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42., storeName=3817a509de9c51aaf4f3e11d1b870d42/cf2, priority=9, startTime=1676015659479; duration=0sec 2023-02-10 07:54:19,534 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:19,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40867] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:47068 deadline: 1676015719596, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3. is not online on jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:19,709 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-10 07:54:19,716 INFO [Listener at localhost/43795] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-10 07:54:19,718 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-10 07:54:19,735 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:19,737 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-10 07:54:19,737 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-10 07:54:19,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:19,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-10 07:54:19,741 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015659741"}]},"ts":"1676015659741"} 2023-02-10 07:54:19,743 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-10 07:54:19,744 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-10 07:54:19,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3817a509de9c51aaf4f3e11d1b870d42, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=39a95a9bfb795ab2b5aa9c3c4cbfb443, UNASSIGN}] 2023-02-10 07:54:19,746 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=39a95a9bfb795ab2b5aa9c3c4cbfb443, UNASSIGN 2023-02-10 07:54:19,746 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=39a95a9bfb795ab2b5aa9c3c4cbfb443, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:19,746 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659746"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015659746"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015659746"}]},"ts":"1676015659746"} 2023-02-10 07:54:19,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 39a95a9bfb795ab2b5aa9c3c4cbfb443, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:19,750 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3817a509de9c51aaf4f3e11d1b870d42, UNASSIGN 2023-02-10 07:54:19,751 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=3817a509de9c51aaf4f3e11d1b870d42, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:19,751 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659750"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015659750"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015659750"}]},"ts":"1676015659750"} 2023-02-10 07:54:19,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 3817a509de9c51aaf4f3e11d1b870d42, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:19,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-10 07:54:19,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 39a95a9bfb795ab2b5aa9c3c4cbfb443, disabling compactions & flushes 2023-02-10 07:54:19,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. after waiting 0 ms 2023-02-10 07:54:19,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,911 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1-top] to archive 2023-02-10 07:54:19,912 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:19,914 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,915 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,916 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,918 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,919 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,920 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,921 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,923 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a-top] to archive 2023-02-10 07:54:19,923 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:19,925 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,926 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,927 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,928 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,929 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,930 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,931 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,935 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:54:19,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:19,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443. 2023-02-10 07:54:19,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 39a95a9bfb795ab2b5aa9c3c4cbfb443: Waiting for close lock at 1676015659900Running coprocessor pre-close hooks at 1676015659900Disabling compacts and flushes for region at 1676015659900Disabling writes for close at 1676015659900Writing region close event to WAL at 1676015659932 (+32 ms)Running coprocessor post-close hooks at 1676015659936 (+4 ms)Closed at 1676015659936 2023-02-10 07:54:19,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:19,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3817a509de9c51aaf4f3e11d1b870d42, disabling compactions & flushes 2023-02-10 07:54:19,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. after waiting 0 ms 2023-02-10 07:54:19,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,938 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=39a95a9bfb795ab2b5aa9c3c4cbfb443, regionState=CLOSED 2023-02-10 07:54:19,938 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659938"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015659938"}]},"ts":"1676015659938"} 2023-02-10 07:54:19,939 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1-bottom] to archive 2023-02-10 07:54:19,940 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:19,942 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-10 07:54:19,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 39a95a9bfb795ab2b5aa9c3c4cbfb443, server=jenkins-hbase4.apache.org,40867,1676015645736 in 193 msec 2023-02-10 07:54:19,943 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=39a95a9bfb795ab2b5aa9c3c4cbfb443, UNASSIGN in 197 msec 2023-02-10 07:54:19,944 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,946 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,947 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,948 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,949 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,951 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a-bottom] to archive 2023-02-10 07:54:19,952 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:19,953 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,954 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,955 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,956 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,957 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,958 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,959 DEBUG [StoreCloser-testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:19,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:54:19,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:19,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42. 2023-02-10 07:54:19,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3817a509de9c51aaf4f3e11d1b870d42: Waiting for close lock at 1676015659937Running coprocessor pre-close hooks at 1676015659937Disabling compacts and flushes for region at 1676015659937Disabling writes for close at 1676015659938 (+1 ms)Writing region close event to WAL at 1676015659960 (+22 ms)Running coprocessor post-close hooks at 1676015659963 (+3 ms)Closed at 1676015659964 (+1 ms) 2023-02-10 07:54:19,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:19,966 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=3817a509de9c51aaf4f3e11d1b870d42, regionState=CLOSED 2023-02-10 07:54:19,966 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015659966"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015659966"}]},"ts":"1676015659966"} 2023-02-10 07:54:19,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-10 07:54:19,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 3817a509de9c51aaf4f3e11d1b870d42, server=jenkins-hbase4.apache.org,40867,1676015645736 in 215 msec 2023-02-10 07:54:19,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-10 07:54:19,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3817a509de9c51aaf4f3e11d1b870d42, UNASSIGN in 224 msec 2023-02-10 07:54:19,972 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015659972"}]},"ts":"1676015659972"} 2023-02-10 07:54:19,973 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-10 07:54:19,975 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-10 07:54:19,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 239 msec 2023-02-10 07:54:20,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-10 07:54:20,044 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-10 07:54:20,044 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-10 07:54:20,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:20,047 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:20,048 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:20,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-10 07:54:20,054 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,054 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:20,054 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:20,056 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/recovered.edits] 2023-02-10 07:54:20,056 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/recovered.edits] 2023-02-10 07:54:20,056 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/recovered.edits] 2023-02-10 07:54:20,060 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/.regioninfo to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/.regioninfo 2023-02-10 07:54:20,060 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/35aa46cd8b524270ab1723e7bdf8ca73 2023-02-10 07:54:20,061 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/.regioninfo to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/.regioninfo 2023-02-10 07:54:20,062 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/36fb54c46f4046ac96e4dedf4a858822 2023-02-10 07:54:20,064 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/3ac7562f8e8f4d1f93b784e394c79a55 2023-02-10 07:54:20,064 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,065 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/35aa46cd8b524270ab1723e7bdf8ca73.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,065 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1 2023-02-10 07:54:20,065 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,066 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/36fb54c46f4046ac96e4dedf4a858822.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,067 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/ad0f629d01294b8e8190834c9c4dff51 2023-02-10 07:54:20,067 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,068 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/3ac7562f8e8f4d1f93b784e394c79a55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,069 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/b580a4cae2294428bb31a97ef4fe01e6 2023-02-10 07:54:20,069 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,070 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/4bd77b67d2eb46c4b56a2f710de7e1d1.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,071 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf1/d210420e123040aa81ab1f37ae7962a9 2023-02-10 07:54:20,071 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,072 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/ad0f629d01294b8e8190834c9c4dff51.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,072 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,074 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/b580a4cae2294428bb31a97ef4fe01e6.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,074 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,074 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/13d521e2dc4946d39ed732b00b65bb55 2023-02-10 07:54:20,076 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/d210420e123040aa81ab1f37ae7962a9.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,076 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/43aa087ffa7b4a2689f2a91e45e64189 2023-02-10 07:54:20,081 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/78cf0a694e26483695ddc4be9e5e44ec 2023-02-10 07:54:20,083 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,083 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/8d058acda94641bcaa5373399d28b5ff 2023-02-10 07:54:20,085 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,086 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/adfb8fa1b6634eca980c99f7a91ea55a 2023-02-10 07:54:20,086 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/13d521e2dc4946d39ed732b00b65bb55.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,088 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,089 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/b6806e3f59ae47509a024dc28708755c 2023-02-10 07:54:20,089 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/43aa087ffa7b4a2689f2a91e45e64189.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,090 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,091 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/78cf0a694e26483695ddc4be9e5e44ec.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,091 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/cf2/cc18ac59a1d24701ac17e97181b74f80 2023-02-10 07:54:20,092 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,093 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/8d058acda94641bcaa5373399d28b5ff.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,094 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,096 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/adfb8fa1b6634eca980c99f7a91ea55a.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,096 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/recovered.edits/85.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3/recovered.edits/85.seqid 2023-02-10 07:54:20,096 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,097 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,098 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/b6806e3f59ae47509a024dc28708755c.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,099 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/3817a509de9c51aaf4f3e11d1b870d42/cf2/cc18ac59a1d24701ac17e97181b74f80.1aecee45528a1e26504546f287f20ff3 2023-02-10 07:54:20,099 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/9bf91a0e67514fff9a16cf669539c04d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf1/9bf91a0e67514fff9a16cf669539c04d 2023-02-10 07:54:20,104 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/2dd35571eb5c4d23b51886b77088f12a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/cf2/2dd35571eb5c4d23b51886b77088f12a 2023-02-10 07:54:20,104 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/a6faf759c77b4b58925ed4fca5ca0f67 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf1/a6faf759c77b4b58925ed4fca5ca0f67 2023-02-10 07:54:20,109 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443/recovered.edits/90.seqid 2023-02-10 07:54:20,112 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:20,112 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b953cd6fcd41439ca4a4f77fe94c446e to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/cf2/b953cd6fcd41439ca4a4f77fe94c446e 2023-02-10 07:54:20,116 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42/recovered.edits/90.seqid 2023-02-10 07:54:20,117 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitWithoutPONR/3817a509de9c51aaf4f3e11d1b870d42 2023-02-10 07:54:20,117 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-10 07:54:20,120 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:20,123 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-10 07:54:20,125 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-10 07:54:20,126 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:20,126 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-10 07:54:20,126 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015660126"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:20,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015660126"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:20,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015660126"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:20,129 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-10 07:54:20,129 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1aecee45528a1e26504546f287f20ff3, NAME => 'testSplitWithoutPONR,,1676015653429.1aecee45528a1e26504546f287f20ff3.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 3817a509de9c51aaf4f3e11d1b870d42, NAME => 'testSplitWithoutPONR,,1676015655411.3817a509de9c51aaf4f3e11d1b870d42.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 39a95a9bfb795ab2b5aa9c3c4cbfb443, NAME => 'testSplitWithoutPONR,41,1676015655411.39a95a9bfb795ab2b5aa9c3c4cbfb443.', STARTKEY => '41', ENDKEY => ''}] 2023-02-10 07:54:20,129 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-10 07:54:20,129 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015660129"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:20,130 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-10 07:54:20,133 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:20,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 89 msec 2023-02-10 07:54:20,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-10 07:54:20,149 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-10 07:54:20,164 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=393 (was 389) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51746 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1154fe46-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:55712 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51668 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:55718 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51658 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:55734 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:55764 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1154fe46-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51618 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51724 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51628 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51590 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:52318 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:55736 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:55696 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:33182 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-973565913_15 at /127.0.0.1:51632 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 859), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 326), ProcessCount=168 (was 168), AvailableMemoryMB=3506 (was 3656) 2023-02-10 07:54:20,178 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=393, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=275, ProcessCount=168, AvailableMemoryMB=3506 2023-02-10 07:54:20,178 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:20,183 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:54:20,183 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:54:20,185 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:20,187 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:54:20,188 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:54:20,189 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,190 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 empty. 2023-02-10 07:54:20,190 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,190 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-10 07:54:20,201 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:20,202 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 324ce5d5ecf6b4389a1c27af6c1de8c6, NAME => 'testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:54:20,209 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:20,210 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 324ce5d5ecf6b4389a1c27af6c1de8c6, disabling compactions & flushes 2023-02-10 07:54:20,210 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,210 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,210 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. after waiting 0 ms 2023-02-10 07:54:20,210 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,210 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,210 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: Waiting for close lock at 1676015660210Disabling compacts and flushes for region at 1676015660210Disabling writes for close at 1676015660210Writing region close event to WAL at 1676015660210Closed at 1676015660210 2023-02-10 07:54:20,212 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:54:20,213 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015660212"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015660212"}]},"ts":"1676015660212"} 2023-02-10 07:54:20,214 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:54:20,215 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:54:20,215 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015660215"}]},"ts":"1676015660215"} 2023-02-10 07:54:20,216 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-10 07:54:20,219 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:20,219 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:20,219 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:20,219 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:20,219 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:20,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN}] 2023-02-10 07:54:20,221 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:20,222 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43281,1676015645662; forceNewPlan=false, retain=false 2023-02-10 07:54:20,372 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:20,373 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,373 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015660373"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015660373"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015660373"}]},"ts":"1676015660373"} 2023-02-10 07:54:20,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:20,528 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,528 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 07:54:20,532 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36036, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 07:54:20,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 324ce5d5ecf6b4389a1c27af6c1de8c6, NAME => 'testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:20,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:20,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:20,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,540 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,542 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1 2023-02-10 07:54:20,542 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1 2023-02-10 07:54:20,542 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324ce5d5ecf6b4389a1c27af6c1de8c6 columnFamilyName cf1 2023-02-10 07:54:20,543 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(310): Store=324ce5d5ecf6b4389a1c27af6c1de8c6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:20,543 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,544 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2 2023-02-10 07:54:20,544 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2 2023-02-10 07:54:20,544 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324ce5d5ecf6b4389a1c27af6c1de8c6 columnFamilyName cf2 2023-02-10 07:54:20,545 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(310): Store=324ce5d5ecf6b4389a1c27af6c1de8c6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:20,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:20,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:20,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:20,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 324ce5d5ecf6b4389a1c27af6c1de8c6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63187315, jitterRate=-0.05843563377857208}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:20,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: Running coprocessor pre-open hook at 1676015660539Writing region info on filesystem at 1676015660539Initializing all the Stores at 1676015660540 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015660540Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015660540Cleaning up temporary data from old regions at 1676015660546 (+6 ms)Cleaning up detritus from prior splits at 1676015660547 (+1 ms)Running coprocessor post-open hooks at 1676015660551 (+4 ms)Region opened successfully at 1676015660551 2023-02-10 07:54:20,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6., pid=43, masterSystemTime=1676015660528 2023-02-10 07:54:20,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,556 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,556 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015660556"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015660556"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015660556"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015660556"}]},"ts":"1676015660556"} 2023-02-10 07:54:20,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-10 07:54:20,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 in 183 msec 2023-02-10 07:54:20,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-10 07:54:20,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN in 339 msec 2023-02-10 07:54:20,562 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:54:20,562 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015660562"}]},"ts":"1676015660562"} 2023-02-10 07:54:20,563 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-10 07:54:20,570 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:54:20,572 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 386 msec 2023-02-10 07:54:20,688 DEBUG [Listener at localhost/43795] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 07:54:20,692 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36050, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 07:54:20,698 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:20,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-10 07:54:20,700 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-10 07:54:20,700 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,700 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-10 07:54:20,700 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:54:20,701 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,701 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,702 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,702 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,702 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:20,703 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:20,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:20,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:20,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:20,704 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,704 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. started... 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:20,704 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:20,705 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:54:20,706 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,707 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,707 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,707 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:20,707 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,707 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:20,718 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/58aa2cf63e60400e99078ab226238847 2023-02-10 07:54:20,732 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/83f443dc5d6241428022b6ee388aea5b 2023-02-10 07:54:20,737 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/58aa2cf63e60400e99078ab226238847 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 2023-02-10 07:54:20,740 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:20,741 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/83f443dc5d6241428022b6ee388aea5b as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b 2023-02-10 07:54:20,746 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:20,747 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 42ms, sequenceid=6, compaction requested=false 2023-02-10 07:54:20,747 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-10 07:54:20,748 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:20,748 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,748 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:20,748 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:20,748 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,748 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:20,748 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:20,754 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:20,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:20,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:20,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-10 07:54:20,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2aeacffa[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:20,757 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:54:20,757 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,758 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:20,758 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:20,758 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:20,758 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:20,758 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,758 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:20,759 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:20,759 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:20,759 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:20,760 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:20,760 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,760 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:20,761 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:20,761 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,761 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:20,761 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,761 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,761 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:20,761 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:20,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:20,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:54:20,765 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-10 07:54:20,765 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:20,765 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:20,765 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-10 07:54:20,765 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:20,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:20,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,768 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,769 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,770 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:20,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,771 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:20,772 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,772 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,775 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,775 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-10 07:54:20,775 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:20,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,776 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:20,776 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,776 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:20,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,776 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,776 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,777 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,777 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,777 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,876 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:20,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:20,894 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:20,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:20,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-10 07:54:20,896 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-10 07:54:20,896 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,896 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-10 07:54:20,896 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:54:20,896 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,896 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,897 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,897 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,897 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,898 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,898 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,898 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:20,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:20,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:20,900 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:20,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,900 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:20,900 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:20,900 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:20,900 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:20,901 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,902 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,902 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:20,904 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:20,904 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:20,904 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,904 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:20,904 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:20,904 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,905 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. started... 2023-02-10 07:54:20,905 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:20,908 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:20,917 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/031481758bf94ff3a6274e0e9deb1193 2023-02-10 07:54:20,930 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/9ddbe5f97afd42f69b6fc940921718f3 2023-02-10 07:54:20,935 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/031481758bf94ff3a6274e0e9deb1193 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 2023-02-10 07:54:20,940 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:20,941 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/9ddbe5f97afd42f69b6fc940921718f3 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 2023-02-10 07:54:20,946 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:20,947 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 42ms, sequenceid=19, compaction requested=false 2023-02-10 07:54:20,947 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:20,947 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:20,947 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:20,947 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:20,947 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,947 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:20,947 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:20,949 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,949 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:20,950 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,950 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:20,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-10 07:54:20,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d8d6b1a[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:20,952 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:54:20,952 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,953 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:20,953 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:20,953 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,954 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:20,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:20,954 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:20,955 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:20,955 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,955 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:20,959 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:20,959 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:20,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:20,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:54:20,962 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-10 07:54:20,963 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:20,963 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:20,963 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-10 07:54:20,963 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:20,964 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,964 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,964 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,964 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,964 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:20,964 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,965 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:20,964 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,965 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,966 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:20,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,968 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:20,968 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:20,969 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,971 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,971 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,971 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,971 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,972 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:20,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-10 07:54:20,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:20,971 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:20,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:20,972 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:20,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,973 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:20,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:20,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,973 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:20,973 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:20,973 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:20,973 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:20,973 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,073 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:21,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:21,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:21,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:21,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-10 07:54:21,090 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,090 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,090 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-10 07:54:21,090 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:54:21,091 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,091 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,092 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,092 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,092 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,092 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,092 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,092 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,093 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,094 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,094 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,094 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:21,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:21,095 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,095 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. started... 2023-02-10 07:54:21,095 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:21,096 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,096 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,097 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,097 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,097 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,097 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,105 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/a41f97e0a767402283bcce0cafea9a60 2023-02-10 07:54:21,118 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/17116e52fc24470999377740267e5f12 2023-02-10 07:54:21,122 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/a41f97e0a767402283bcce0cafea9a60 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 2023-02-10 07:54:21,126 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:21,126 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/17116e52fc24470999377740267e5f12 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 2023-02-10 07:54:21,130 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:21,131 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 36ms, sequenceid=32, compaction requested=true 2023-02-10 07:54:21,131 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:21,131 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,132 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,132 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-10 07:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5061de97[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:21,134 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:54:21,135 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,136 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,136 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,137 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,137 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,139 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,139 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:21,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:54:21,143 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-10 07:54:21,144 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:21,144 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:21,144 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-10 07:54:21,144 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:21,145 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,145 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,145 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,145 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,145 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,145 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,146 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,146 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,146 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,147 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,149 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,149 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,149 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:21,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,155 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,155 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,156 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:21,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,156 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:21,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,157 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,157 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,157 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,157 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,157 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,157 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,157 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,257 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:21,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:21,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:21,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:21,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-10 07:54:21,274 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,274 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,275 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-10 07:54:21,275 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:54:21,275 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,275 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,276 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,276 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,276 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,276 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,276 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,277 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:21,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,278 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,278 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,278 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,278 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,278 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:21,279 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:21,279 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,279 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. started... 2023-02-10 07:54:21,279 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:21,280 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,281 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,281 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,281 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,281 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,281 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,292 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/efa9af1d69134ccdab2ae72960af1bf9 2023-02-10 07:54:21,304 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/afcd3bd46c8f46518831eb9f1c441581 2023-02-10 07:54:21,308 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/efa9af1d69134ccdab2ae72960af1bf9 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 2023-02-10 07:54:21,312 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:21,313 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/afcd3bd46c8f46518831eb9f1c441581 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 2023-02-10 07:54:21,317 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:21,318 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 39ms, sequenceid=45, compaction requested=true 2023-02-10 07:54:21,318 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:21,318 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,318 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,321 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,321 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-10 07:54:21,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@386b5e2f[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:21,323 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:54:21,323 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,324 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,324 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,324 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:21,324 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,324 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,324 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:21,324 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,325 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,324 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:21,325 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:21,326 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,326 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,326 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,326 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,326 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,326 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,326 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,327 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,327 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,327 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:21,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:54:21,329 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-10 07:54:21,329 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:21,329 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:21,329 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-10 07:54:21,329 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:21,331 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,331 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,332 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,331 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,332 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,331 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,332 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,332 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,332 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,333 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,334 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,335 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,335 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,338 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,338 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,338 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,338 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,338 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,338 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,338 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,338 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:21,338 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,339 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,339 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,339 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,339 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,339 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,339 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:21,339 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:21,439 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:21,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:21,455 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:21,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:21,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-10 07:54:21,457 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,457 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,457 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-10 07:54:21,457 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:54:21,457 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,458 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,459 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,459 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,459 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:21,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,461 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,461 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,461 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,461 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,461 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,461 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,461 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:21,461 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:21,462 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,462 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. started... 2023-02-10 07:54:21,462 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,472 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/48ad9e1ea7eb43a083da70c85a88cce4 2023-02-10 07:54:21,487 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/2afc38a93db94ad78eb64bd0f3d7de8c 2023-02-10 07:54:21,491 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/48ad9e1ea7eb43a083da70c85a88cce4 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 2023-02-10 07:54:21,495 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:21,496 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/2afc38a93db94ad78eb64bd0f3d7de8c as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c 2023-02-10 07:54:21,500 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:21,501 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 39ms, sequenceid=58, compaction requested=true 2023-02-10 07:54:21,501 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:21,501 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,501 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:21,501 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:21,501 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,501 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,501 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,504 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,504 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,504 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,504 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-10 07:54:21,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4402c3c7[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:21,506 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:54:21,506 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,507 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,507 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:21,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,507 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,507 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,512 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,513 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,513 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,513 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:21,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:54:21,516 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-10 07:54:21,516 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:21,516 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:21,516 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-10 07:54:21,516 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:21,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,519 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,519 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,521 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,521 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,521 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,524 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,524 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,524 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,525 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,525 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:21,525 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,525 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,525 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,525 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,525 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,625 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:21,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:21,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:21,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:21,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-10 07:54:21,642 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,642 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,642 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-10 07:54:21,642 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:54:21,643 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,643 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,644 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,644 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,644 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,644 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,644 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,644 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:21,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,646 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:21,646 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-10 07:54:21,646 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:21,646 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. started... 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,646 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,647 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,647 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:21,649 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,650 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,650 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,650 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,650 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,650 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,657 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/c3980355c4654bf5830e49675ee808bb 2023-02-10 07:54:21,669 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/340d30fe12674218ac7a8d26acf267d6 2023-02-10 07:54:21,673 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/c3980355c4654bf5830e49675ee808bb as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb 2023-02-10 07:54:21,677 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:21,678 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/340d30fe12674218ac7a8d26acf267d6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 2023-02-10 07:54:21,681 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:21,682 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 36ms, sequenceid=71, compaction requested=true 2023-02-10 07:54:21,682 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:21,682 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:21,682 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:21,683 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:21,683 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,683 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-10 07:54:21,683 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-10 07:54:21,684 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,684 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,684 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,685 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:21,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-10 07:54:21,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32a1a5e6[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:21,686 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:54:21,686 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,688 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,688 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:21,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:21,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:21,690 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,690 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,690 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,691 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,691 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,691 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,691 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,692 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-10 07:54:21,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,692 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,692 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-10 07:54:21,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:21,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:54:21,695 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-10 07:54:21,695 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:21,695 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:21,695 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-10 07:54:21,695 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:21,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:21,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,697 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,698 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,698 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:21,700 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-10 07:54:21,700 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,701 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,709 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-10 07:54:21,709 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,709 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:21,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:21,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,709 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:21,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-10 07:54:21,710 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-10 07:54:21,810 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:21,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:22,070 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-10 07:54:22,072 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:22,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:22,075 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:22,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:22,323 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,323 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:22,323 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:22,323 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:22,324 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-10 07:54:22,324 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:22,324 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:22,324 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:22,324 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:22,325 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:22,326 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-10 07:54:22,326 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:22,327 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:22,327 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:22,327 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:22,328 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,328 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:22,328 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,329 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,329 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,329 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:22,330 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,330 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,330 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:22,330 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,330 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:22,330 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,331 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,331 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,331 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:22,331 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:22,332 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:22,332 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015692332 2023-02-10 07:54:22,334 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:22,334 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,336 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:22,336 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:22,336 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:22,336 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015722336 2023-02-10 07:54:22,336 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015782336 2023-02-10 07:54:22,336 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:54:22,337 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:22,338 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:22,338 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:22,338 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:22,587 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,587 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:22,587 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:22,587 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:22,588 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:22,588 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:22,588 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:22,588 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:22,588 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:22,589 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:22,590 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-10 07:54:22,590 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:22,591 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:22,591 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:22,591 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:22,591 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,592 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:22,592 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,592 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,592 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,592 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:22,592 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,593 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,593 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:22,593 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,593 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:22,593 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,593 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,593 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,594 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-10 07:54:22,594 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:22,595 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:22,595 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015692595 2023-02-10 07:54:22,596 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:22,597 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,600 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:22,600 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:22,600 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:22,600 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015722600 2023-02-10 07:54:22,600 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015782600 2023-02-10 07:54:22,600 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:22,601 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:22,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN}] 2023-02-10 07:54:22,603 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:22,603 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:22,603 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:22,851 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,851 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:22,851 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:22,851 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:22,853 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:22,853 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:22,854 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:22,854 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:22,854 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:22,854 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:22,855 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-10 07:54:22,856 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:22,856 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:22,856 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:22,856 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:22,857 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,857 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:22,857 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,857 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:22,858 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,858 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:22,858 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,859 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:22,859 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:22,859 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,860 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:22,860 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,860 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,860 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,860 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:22,860 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:22,861 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:22,861 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015692861 2023-02-10 07:54:22,862 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:22,862 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:22,865 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:22,865 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:22,865 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:22,865 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015722865 2023-02-10 07:54:22,865 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015782865 2023-02-10 07:54:22,865 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:22,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN}] 2023-02-10 07:54:22,865 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:22,867 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:22,868 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:22,868 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015662868"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015662868"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015662868"}]},"ts":"1676015662868"} 2023-02-10 07:54:22,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:22,870 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:22,870 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:22,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:22,870 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:23,115 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,116 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:23,116 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:23,116 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:23,117 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:23,117 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:23,117 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:23,117 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:23,117 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:23,118 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:23,119 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:23,119 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-10 07:54:23,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:23,120 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:23,120 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:23,120 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:23,120 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,121 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:23,121 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,121 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,121 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,122 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:23,122 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,122 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,122 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:23,123 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,123 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,123 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:23,123 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,123 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,123 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,124 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:23,124 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,124 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:23,124 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:23,125 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:23,125 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:23,125 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015693125 2023-02-10 07:54:23,125 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:23,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,129 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:23,129 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:23,129 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:23,129 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015723129 2023-02-10 07:54:23,129 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015783129 2023-02-10 07:54:23,129 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:23,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:23,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015663130"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015663130"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015663130"}]},"ts":"1676015663130"} 2023-02-10 07:54:23,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:23,132 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:23,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 324ce5d5ecf6b4389a1c27af6c1de8c6, disabling compactions & flushes 2023-02-10 07:54:23,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:23,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:23,285 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. after waiting 0 ms 2023-02-10 07:54:23,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:23,286 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-10 07:54:23,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/d27546f8bbd740e9a5caafa5af3ca647 2023-02-10 07:54:23,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/8ddb0908953c430fb16a71ed9d86ce02 2023-02-10 07:54:23,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/d27546f8bbd740e9a5caafa5af3ca647 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 2023-02-10 07:54:23,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:54:23,321 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/8ddb0908953c430fb16a71ed9d86ce02 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 2023-02-10 07:54:23,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:54:23,325 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 39ms, sequenceid=82, compaction requested=true 2023-02-10 07:54:23,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-10 07:54:23,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:23,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:23,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: Waiting for close lock at 1676015663285Running coprocessor pre-close hooks at 1676015663285Disabling compacts and flushes for region at 1676015663285Disabling writes for close at 1676015663286 (+1 ms)Obtaining lock to block concurrent updates at 1676015663286Preparing flush snapshotting stores in 324ce5d5ecf6b4389a1c27af6c1de8c6 at 1676015663286Finished memstore snapshotting testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676015663286Flushing stores of testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. at 1676015663287 (+1 ms)Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf1: creating writer at 1676015663288 (+1 ms)Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf1: appending metadata at 1676015663290 (+2 ms)Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf1: closing flushed file at 1676015663290Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf2: creating writer at 1676015663301 (+11 ms)Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf2: appending metadata at 1676015663303 (+2 ms)Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf2: closing flushed file at 1676015663303Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf1: reopening flushed file at 1676015663317 (+14 ms)Flushing 324ce5d5ecf6b4389a1c27af6c1de8c6/cf2: reopening flushed file at 1676015663321 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 324ce5d5ecf6b4389a1c27af6c1de8c6 in 39ms, sequenceid=82, compaction requested=true at 1676015663325 (+4 ms)Writing region close event to WAL at 1676015663327 (+2 ms)Running coprocessor post-close hooks at 1676015663330 (+3 ms)Closed at 1676015663330 2023-02-10 07:54:23,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,333 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED 2023-02-10 07:54:23,333 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015663333"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015663333"}]},"ts":"1676015663333"} 2023-02-10 07:54:23,334 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:23,334 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:23,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:23,334 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:23,380 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,380 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:23,380 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:23,380 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:23,381 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:23,381 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:23,381 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:23,381 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:23,381 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:23,382 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:23,383 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:23,383 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-10 07:54:23,383 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-10 07:54:23,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:23,384 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:23,384 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:23,384 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:23,384 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,385 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:23,385 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,385 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,385 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,386 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:23,386 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,386 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,387 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:23,387 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,387 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,387 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:23,387 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:23,388 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,388 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,388 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,388 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:23,388 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,388 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:23,388 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:23,389 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:23,389 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:23,390 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:23,390 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015693390 2023-02-10 07:54:23,390 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:23,391 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:23,398 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:23,399 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:23,399 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:23,399 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015723399 2023-02-10 07:54:23,399 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015783399 2023-02-10 07:54:23,399 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 9 msec 2023-02-10 07:54:23,399 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:23,400 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015663399"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015663399"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015663399"}]},"ts":"1676015663399"} 2023-02-10 07:54:23,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:23,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-10 07:54:23,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 in 269 msec 2023-02-10 07:54:23,405 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:23,405 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:23,407 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:23,650 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:23,650 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:23,651 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:23,652 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:23,652 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-10 07:54:23,652 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:23,653 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:23,653 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:23,653 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:23,654 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:23,654 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,654 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:23,654 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,655 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,655 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,655 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:23,655 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,655 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,656 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:23,657 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,657 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,657 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:23,657 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:23,657 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,657 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,657 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,657 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:23,657 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:23,657 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:54:23,658 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:23,658 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:23,658 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:23,659 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015693659 2023-02-10 07:54:23,659 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:23,660 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,662 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:23,662 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:23,662 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:23,662 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015723662 2023-02-10 07:54:23,662 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015783662 2023-02-10 07:54:23,663 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:23,663 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,664 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:23,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-10 07:54:23,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN in 799 msec 2023-02-10 07:54:23,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:23,670 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:23,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:23,913 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:23,913 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:23,914 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:23,915 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:23,915 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:23,915 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:23,915 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:23,916 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:23,916 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:23,916 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:23,917 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,917 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:23,917 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,917 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:23,918 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,918 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:23,918 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,918 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:23,918 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:23,919 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,919 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:23,919 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:23,920 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:23,920 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:23,920 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:23,920 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:23,921 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:23,921 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:23,921 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015693921 2023-02-10 07:54:23,922 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:23,923 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,925 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:23,926 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:23,926 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:23,926 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015723926 2023-02-10 07:54:23,926 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015783926 2023-02-10 07:54:23,926 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:23,926 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,926 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,926 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:23,931 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:23,942 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=324ce5d5ecf6b4389a1c27af6c1de8c6, threads=14 2023-02-10 07:54:23,943 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,943 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,943 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,943 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,943 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,948 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,948 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,948 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,948 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,952 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,952 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,952 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,952 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:23,952 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,122 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,132 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,137 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,143 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,150 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,183 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,184 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,186 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,191 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,192 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,544 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,545 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,547 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,548 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,548 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 324ce5d5ecf6b4389a1c27af6c1de8c6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:54:24,590 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:24,590 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:24,590 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:24,668 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:54:24,676 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,676 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:24,677 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:24,677 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:24,678 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:24,678 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:24,678 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:24,678 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:24,678 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:24,679 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:24,680 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-10 07:54:24,681 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:24,681 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:24,681 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:24,681 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:24,682 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:24,682 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:24,682 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:24,683 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:24,683 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:24,683 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:24,683 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:24,684 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:24,684 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:24,684 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:24,685 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:24,685 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,685 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:24,686 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:24,686 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:24,686 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,686 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,686 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,686 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:24,686 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:24,686 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:54:24,687 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:24,688 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:24,689 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015694689 2023-02-10 07:54:24,691 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,691 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:24,693 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:24,693 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:24,693 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:24,694 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015724693 2023-02-10 07:54:24,694 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015784694 2023-02-10 07:54:24,694 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:24,705 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=324ce5d5ecf6b4389a1c27af6c1de8c6, threads=14 2023-02-10 07:54:24,706 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,706 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,706 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,706 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,706 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,708 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,708 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,712 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,712 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,712 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,713 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,713 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,714 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,714 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,824 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,847 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,888 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,889 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,893 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,912 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,913 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,928 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,932 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,933 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,934 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,935 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,936 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,936 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 for region: 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:24,936 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 324ce5d5ecf6b4389a1c27af6c1de8c6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:54:24,975 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:24,981 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/f2991a54330cb9cf607bcfb8f78193ec/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:54:24,982 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/7a0bbd91d8a69cf2357c5f560a7502d1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:54:24,982 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:24,982 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:24,982 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:24,983 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:25,194 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,194 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:25,194 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:25,194 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:25,195 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:25,196 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:25,196 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:25,196 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:25,196 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:25,197 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:25,198 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-10 07:54:25,198 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:25,198 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:25,198 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:25,198 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:25,199 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:25,199 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:25,199 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,199 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:25,199 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,200 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,200 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,200 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:25,200 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,201 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,201 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:25,201 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,201 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:25,202 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:25,203 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:25,204 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,204 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,204 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,204 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,204 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,204 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-10 07:54:25,205 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:25,205 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:25,206 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015695206 2023-02-10 07:54:25,206 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:25,207 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,209 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:25,209 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:25,209 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:25,210 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015725210 2023-02-10 07:54:25,210 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015785210 2023-02-10 07:54:25,210 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:25,212 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:25,215 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:25,215 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:25,215 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:25,460 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,460 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:25,461 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:25,461 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:25,461 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:25,461 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:25,461 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:25,461 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:25,461 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:25,462 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:25,462 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-10 07:54:25,463 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:25,463 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:25,463 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:25,463 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:25,463 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:25,464 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:25,464 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,464 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:25,464 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,464 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,465 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,465 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:25,465 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,466 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,466 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:25,466 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,467 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:25,467 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:25,467 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:25,467 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,467 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,467 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,467 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,467 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,468 DEBUG [Listener at localhost/43795] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,468 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:25,468 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:25,469 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:25,469 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015695469 2023-02-10 07:54:25,469 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:25,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,474 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:25,474 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:25,474 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:25,474 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015725474 2023-02-10 07:54:25,474 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015785474 2023-02-10 07:54:25,474 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:25,474 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:25,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:25,486 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-10 07:54:25,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:25,487 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:25,725 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,725 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:25,725 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:25,725 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:25,726 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:25,726 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:25,726 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:25,726 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:25,726 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:25,727 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:25,728 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-10 07:54:25,728 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:25,728 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:25,729 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:25,729 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:25,729 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:25,729 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:25,730 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,730 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:25,730 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,730 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,730 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,730 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:25,731 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,731 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,731 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:25,732 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,732 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:25,732 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:25,732 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:25,733 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,733 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,733 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,733 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,733 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,733 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:25,733 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:25,734 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:25,734 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015695734 2023-02-10 07:54:25,734 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:25,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,739 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:25,739 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:25,739 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:25,739 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015725739 2023-02-10 07:54:25,739 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015785739 2023-02-10 07:54:25,739 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-10 07:54:25,740 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:25,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:25,742 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-10 07:54:25,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:25,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:25,991 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,991 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:25,991 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:25,991 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:25,992 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:25,992 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:25,992 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:25,992 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:25,992 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:25,993 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:25,993 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-10 07:54:25,994 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:25,994 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:25,994 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:25,995 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:25,995 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:25,995 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:25,995 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,996 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:25,996 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,996 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:25,997 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,997 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:25,997 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,997 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:25,997 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:25,998 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,998 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:25,998 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:25,998 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:25,998 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,998 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,998 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:25,999 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,999 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:25,999 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:25,999 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:26,000 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:26,000 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015696000 2023-02-10 07:54:26,001 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:26,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,004 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:26,004 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:26,004 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:26,004 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015726004 2023-02-10 07:54:26,004 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015786004 2023-02-10 07:54:26,005 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:26,005 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:26,007 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:26,007 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-10 07:54:26,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:26,007 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:26,255 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,255 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:26,255 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:26,255 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:26,256 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:26,256 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:26,256 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:26,256 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:26,256 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:26,257 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:26,258 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-10 07:54:26,258 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:26,258 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:26,258 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:26,258 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:26,259 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:26,259 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:26,259 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,259 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:26,259 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,260 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,260 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,260 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:26,260 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,260 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,261 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:26,261 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,261 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:26,262 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:26,262 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:26,262 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,262 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,262 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,262 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:26,262 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:26,262 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:26,263 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:26,263 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:26,264 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015696264 2023-02-10 07:54:26,265 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:26,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,267 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:26,268 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:26,268 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:26,268 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015726268 2023-02-10 07:54:26,268 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015786268 2023-02-10 07:54:26,268 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:26,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:26,271 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN] 2023-02-10 07:54:26,271 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:26,271 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-10 07:54:26,271 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:26,271 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:26,518 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,518 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:26,518 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:26,518 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:26,521 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:26,521 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:26,521 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:26,521 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:26,521 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:26,522 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:26,522 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-10 07:54:26,522 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:54:26,523 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:54:26,523 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-10 07:54:26,523 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:26,523 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:26,523 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:26,523 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:26,524 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,524 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:26,524 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,524 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,524 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,525 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:26,525 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,525 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,525 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:26,527 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,527 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:26,527 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:26,527 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:26,527 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:26,527 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,527 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,527 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,528 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:26,528 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:26,528 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:26,528 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:54:26,529 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:26,529 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:26,529 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:26,530 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015696530 2023-02-10 07:54:26,530 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:26,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,532 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:26,534 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:26,534 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:26,534 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:26,535 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015726535 2023-02-10 07:54:26,535 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015786535 2023-02-10 07:54:26,535 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:26,535 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:26,539 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:26,539 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-10 07:54:26,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:26,540 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:26,785 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,785 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:26,785 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:26,785 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:26,785 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:26,785 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:26,785 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:26,785 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:26,786 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:26,786 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:26,787 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:54:26,787 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-10 07:54:26,787 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:26,788 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:26,788 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:26,788 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:26,788 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,788 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:26,788 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,789 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:26,789 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,789 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:26,789 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,789 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:26,790 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:26,790 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,790 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:26,790 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:26,790 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,790 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,790 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,790 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:26,790 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-10 07:54:26,791 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:26,791 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:26,792 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:26,792 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015696792 2023-02-10 07:54:26,792 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:26,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:26,795 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:26,797 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:26,797 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:26,797 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:26,797 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015726797 2023-02-10 07:54:26,797 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015786797 2023-02-10 07:54:26,797 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:26,798 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:26,800 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:26,800 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-10 07:54:26,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:26,800 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:27,048 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:27,048 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:27,049 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:27,050 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-10 07:54:27,050 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-10 07:54:27,050 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:27,050 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:27,050 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:27,051 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:27,051 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,051 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:27,051 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,051 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,052 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,052 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:27,052 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,052 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,052 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:27,052 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,053 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,053 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:27,053 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,053 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,053 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,053 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:27,053 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-10 07:54:27,054 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:27,054 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:27,054 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:27,054 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015697054 2023-02-10 07:54:27,055 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:27,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,057 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,059 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:27,059 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:27,059 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:27,059 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015727059 2023-02-10 07:54:27,059 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015787059 2023-02-10 07:54:27,059 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:54:27,059 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:27,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:27,062 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-10 07:54:27,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:27,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:27,310 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,310 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:27,310 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:27,310 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:27,310 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:27,310 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:27,310 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:27,310 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:27,311 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:27,311 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:27,312 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-10 07:54:27,313 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-10 07:54:27,313 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:27,313 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:27,313 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:27,314 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:27,314 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,314 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:27,314 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,315 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,315 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,315 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:27,315 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,316 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,316 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:27,316 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,316 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,317 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:27,317 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,317 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,317 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,317 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:54:27,317 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-10 07:54:27,318 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:27,319 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:27,319 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:27,319 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015697319 2023-02-10 07:54:27,320 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:27,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,323 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,325 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:27,325 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:27,325 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:27,325 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015727325 2023-02-10 07:54:27,325 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015787325 2023-02-10 07:54:27,325 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:54:27,326 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:54:27,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 exec-time=5.2540 sec 2023-02-10 07:54:27,328 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,329 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43281,1676015645662; forceNewPlan=false, retain=true 2023-02-10 07:54:27,479 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:27,480 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:27,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015667480"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015667480"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015667480"}]},"ts":"1676015667480"} 2023-02-10 07:54:27,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:27,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:54:27,482 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:54:27,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:54:27,576 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:27,576 WARN [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-10 07:54:27,576 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:54:27,576 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:54:27,576 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:54:27,576 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:54:27,576 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:54:27,576 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:54:27,576 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:54:27,577 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:54:27,577 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:54:27,578 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-10 07:54:27,578 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:54:27,579 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:54:27,579 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:54:27,579 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:54:27,579 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,579 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:54:27,579 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,580 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:54:27,580 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,580 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:54:27,580 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,580 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:54:27,581 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:27,581 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:54:27,581 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,581 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:54:27,581 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,581 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-10 07:54:27,582 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:54:27,582 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:54:27,582 INFO [Listener at localhost/43795] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6 to restore RIT 2023-02-10 07:54:27,583 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015697583 2023-02-10 07:54:27,584 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:54:27,584 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:54:27,586 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:54:27,586 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,43281,1676015645662, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662, openSeqNum=2 2023-02-10 07:54:27,586 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:54:27,586 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015727586 2023-02-10 07:54:27,586 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015787586 2023-02-10 07:54:27,586 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:54:27,586 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:27,586 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015667586"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015667586"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015667586"}]},"ts":"1676015667586"} 2023-02-10 07:54:27,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:27,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:27,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 324ce5d5ecf6b4389a1c27af6c1de8c6, NAME => 'testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:27,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:27,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:27,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,745 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,746 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1 2023-02-10 07:54:27,746 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1 2023-02-10 07:54:27,746 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324ce5d5ecf6b4389a1c27af6c1de8c6 columnFamilyName cf1 2023-02-10 07:54:27,752 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 2023-02-10 07:54:27,755 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 2023-02-10 07:54:27,757 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 2023-02-10 07:54:27,760 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 2023-02-10 07:54:27,763 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb 2023-02-10 07:54:27,765 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 2023-02-10 07:54:27,768 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 2023-02-10 07:54:27,769 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(310): Store=324ce5d5ecf6b4389a1c27af6c1de8c6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:27,769 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,770 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2 2023-02-10 07:54:27,770 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2 2023-02-10 07:54:27,770 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324ce5d5ecf6b4389a1c27af6c1de8c6 columnFamilyName cf2 2023-02-10 07:54:27,776 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 2023-02-10 07:54:27,779 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c 2023-02-10 07:54:27,782 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 2023-02-10 07:54:27,785 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b 2023-02-10 07:54:27,788 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 2023-02-10 07:54:27,792 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 2023-02-10 07:54:27,794 DEBUG [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 2023-02-10 07:54:27,795 INFO [StoreOpener-324ce5d5ecf6b4389a1c27af6c1de8c6-1] regionserver.HStore(310): Store=324ce5d5ecf6b4389a1c27af6c1de8c6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:27,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.splits 2023-02-10 07:54:27,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:27,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:27,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 324ce5d5ecf6b4389a1c27af6c1de8c6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60894328, jitterRate=-0.09260380268096924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:27,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: Running coprocessor pre-open hook at 1676015667744Writing region info on filesystem at 1676015667744Initializing all the Stores at 1676015667744Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015667744Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015667745 (+1 ms)Cleaning up temporary data from old regions at 1676015667797 (+52 ms)Cleaning up detritus from prior splits at 1676015667797Running coprocessor post-open hooks at 1676015667800 (+3 ms)Region opened successfully at 1676015667800 2023-02-10 07:54:27,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6., pid=48, masterSystemTime=1676015667740 2023-02-10 07:54:27,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:27,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:27,802 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:27,802 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:27,803 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-10 07:54:27,803 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-10 07:54:27,804 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1912): 324ce5d5ecf6b4389a1c27af6c1de8c6/cf1 is initiating minor compaction (all files) 2023-02-10 07:54:27,804 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1912): 324ce5d5ecf6b4389a1c27af6c1de8c6/cf2 is initiating minor compaction (all files) 2023-02-10 07:54:27,804 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 324ce5d5ecf6b4389a1c27af6c1de8c6/cf1 in testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:27,804 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 324ce5d5ecf6b4389a1c27af6c1de8c6/cf2 in testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:27,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:27,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:27,804 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp, totalSize=34.9 K 2023-02-10 07:54:27,804 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp, totalSize=34.9 K 2023-02-10 07:54:27,805 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 83f443dc5d6241428022b6ee388aea5b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015660692 2023-02-10 07:54:27,805 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting 58aa2cf63e60400e99078ab226238847, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015660692 2023-02-10 07:54:27,805 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:27,805 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015667805"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015667805"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015667805"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015667805"}]},"ts":"1676015667805"} 2023-02-10 07:54:27,805 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 9ddbe5f97afd42f69b6fc940921718f3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015660878 2023-02-10 07:54:27,805 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting 031481758bf94ff3a6274e0e9deb1193, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015660878 2023-02-10 07:54:27,806 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 17116e52fc24470999377740267e5f12, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015661075 2023-02-10 07:54:27,806 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting a41f97e0a767402283bcce0cafea9a60, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015661075 2023-02-10 07:54:27,806 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting afcd3bd46c8f46518831eb9f1c441581, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015661259 2023-02-10 07:54:27,806 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting efa9af1d69134ccdab2ae72960af1bf9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015661259 2023-02-10 07:54:27,806 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 2afc38a93db94ad78eb64bd0f3d7de8c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015661441 2023-02-10 07:54:27,807 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting 48ad9e1ea7eb43a083da70c85a88cce4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015661441 2023-02-10 07:54:27,807 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 340d30fe12674218ac7a8d26acf267d6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015661627 2023-02-10 07:54:27,807 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting c3980355c4654bf5830e49675ee808bb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015661627 2023-02-10 07:54:27,807 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 8ddb0908953c430fb16a71ed9d86ce02, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015661812 2023-02-10 07:54:27,807 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting d27546f8bbd740e9a5caafa5af3ca647, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015661812 2023-02-10 07:54:27,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-10 07:54:27,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 in 219 msec 2023-02-10 07:54:27,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN in 1.5390 sec 2023-02-10 07:54:27,833 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] throttle.PressureAwareThroughputController(145): 324ce5d5ecf6b4389a1c27af6c1de8c6#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:27,836 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] throttle.PressureAwareThroughputController(145): 324ce5d5ecf6b4389a1c27af6c1de8c6#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:27,837 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-10 07:54:27,840 DEBUG [Listener at localhost/43795] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-10 07:54:27,842 INFO [Listener at localhost/43795] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-10 07:54:27,845 INFO [Listener at localhost/43795] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-10 07:54:27,846 INFO [Listener at localhost/43795] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-10 07:54:27,846 INFO [Listener at localhost/43795] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-10 07:54:27,855 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf2/24686013b1564b32a7cc370a35117c3a as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/24686013b1564b32a7cc370a35117c3a 2023-02-10 07:54:27,863 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 324ce5d5ecf6b4389a1c27af6c1de8c6/cf2 of 324ce5d5ecf6b4389a1c27af6c1de8c6 into 24686013b1564b32a7cc370a35117c3a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:27,863 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:27,863 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6., storeName=324ce5d5ecf6b4389a1c27af6c1de8c6/cf2, priority=9, startTime=1676015667801; duration=0sec 2023-02-10 07:54:27,864 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:27,866 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/.tmp/cf1/06f40894307d4a188e800186a9a25eba as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/06f40894307d4a188e800186a9a25eba 2023-02-10 07:54:27,874 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 324ce5d5ecf6b4389a1c27af6c1de8c6/cf1 of 324ce5d5ecf6b4389a1c27af6c1de8c6 into 06f40894307d4a188e800186a9a25eba(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:27,874 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: 2023-02-10 07:54:27,875 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6., storeName=324ce5d5ecf6b4389a1c27af6c1de8c6/cf1, priority=9, startTime=1676015667801; duration=0sec 2023-02-10 07:54:27,875 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:27,881 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:27,883 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-10 07:54:27,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-10 07:54:27,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:27,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-10 07:54:27,887 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015667887"}]},"ts":"1676015667887"} 2023-02-10 07:54:27,888 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-10 07:54:27,890 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-10 07:54:27,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN}] 2023-02-10 07:54:27,891 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN 2023-02-10 07:54:27,892 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:27,892 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015667892"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015667892"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015667892"}]},"ts":"1676015667892"} 2023-02-10 07:54:27,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:27,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-10 07:54:28,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:28,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 324ce5d5ecf6b4389a1c27af6c1de8c6, disabling compactions & flushes 2023-02-10 07:54:28,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:28,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:28,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. after waiting 0 ms 2023-02-10 07:54:28,045 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:28,046 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647] to archive 2023-02-10 07:54:28,047 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:28,048 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/58aa2cf63e60400e99078ab226238847 2023-02-10 07:54:28,050 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/031481758bf94ff3a6274e0e9deb1193 2023-02-10 07:54:28,051 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/a41f97e0a767402283bcce0cafea9a60 2023-02-10 07:54:28,052 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/efa9af1d69134ccdab2ae72960af1bf9 2023-02-10 07:54:28,053 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/48ad9e1ea7eb43a083da70c85a88cce4 2023-02-10 07:54:28,054 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/c3980355c4654bf5830e49675ee808bb 2023-02-10 07:54:28,055 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/d27546f8bbd740e9a5caafa5af3ca647 2023-02-10 07:54:28,059 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02] to archive 2023-02-10 07:54:28,060 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:28,061 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/83f443dc5d6241428022b6ee388aea5b 2023-02-10 07:54:28,062 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/9ddbe5f97afd42f69b6fc940921718f3 2023-02-10 07:54:28,063 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/17116e52fc24470999377740267e5f12 2023-02-10 07:54:28,064 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/afcd3bd46c8f46518831eb9f1c441581 2023-02-10 07:54:28,065 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/2afc38a93db94ad78eb64bd0f3d7de8c 2023-02-10 07:54:28,067 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/340d30fe12674218ac7a8d26acf267d6 2023-02-10 07:54:28,068 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/8ddb0908953c430fb16a71ed9d86ce02 2023-02-10 07:54:28,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:54:28,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:28,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6. 2023-02-10 07:54:28,072 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 324ce5d5ecf6b4389a1c27af6c1de8c6: Waiting for close lock at 1676015668045Running coprocessor pre-close hooks at 1676015668045Disabling compacts and flushes for region at 1676015668045Disabling writes for close at 1676015668045Writing region close event to WAL at 1676015668069 (+24 ms)Running coprocessor post-close hooks at 1676015668072 (+3 ms)Closed at 1676015668072 2023-02-10 07:54:28,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:28,074 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=324ce5d5ecf6b4389a1c27af6c1de8c6, regionState=CLOSED 2023-02-10 07:54:28,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015668074"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015668074"}]},"ts":"1676015668074"} 2023-02-10 07:54:28,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-10 07:54:28,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 324ce5d5ecf6b4389a1c27af6c1de8c6, server=jenkins-hbase4.apache.org,43281,1676015645662 in 182 msec 2023-02-10 07:54:28,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-10 07:54:28,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, UNASSIGN in 187 msec 2023-02-10 07:54:28,079 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015668079"}]},"ts":"1676015668079"} 2023-02-10 07:54:28,080 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-10 07:54:28,081 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-10 07:54:28,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 198 msec 2023-02-10 07:54:28,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-10 07:54:28,189 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-10 07:54:28,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-10 07:54:28,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:28,192 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:28,193 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:28,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-10 07:54:28,196 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:28,198 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/recovered.edits] 2023-02-10 07:54:28,200 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/06f40894307d4a188e800186a9a25eba to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf1/06f40894307d4a188e800186a9a25eba 2023-02-10 07:54:28,202 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/24686013b1564b32a7cc370a35117c3a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/cf2/24686013b1564b32a7cc370a35117c3a 2023-02-10 07:54:28,205 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6/recovered.edits/90.seqid 2023-02-10 07:54:28,205 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackAndDoubleExecution/324ce5d5ecf6b4389a1c27af6c1de8c6 2023-02-10 07:54:28,205 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-10 07:54:28,208 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:28,210 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-10 07:54:28,212 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-10 07:54:28,213 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:28,213 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-10 07:54:28,213 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015668213"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:28,214 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-10 07:54:28,214 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 324ce5d5ecf6b4389a1c27af6c1de8c6, NAME => 'testRollbackAndDoubleExecution,,1676015660184.324ce5d5ecf6b4389a1c27af6c1de8c6.', STARTKEY => '', ENDKEY => ''}] 2023-02-10 07:54:28,214 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-10 07:54:28,214 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015668214"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:28,216 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-10 07:54:28,223 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:54:28,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 34 msec 2023-02-10 07:54:28,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-10 07:54:28,295 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-10 07:54:28,310 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=397 (was 393) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:59866 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:51746 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:55712 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:51668 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:55718 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:55764 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:59852 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:51724 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:52532 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:51628 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:51590 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:43281-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:52318 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:55736 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:55696 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:59824 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:33182 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=882 (was 855) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=252 (was 275), ProcessCount=168 (was 168), AvailableMemoryMB=3486 (was 3506) 2023-02-10 07:54:28,323 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=397, OpenFileDescriptor=882, MaxFileDescriptor=60000, SystemLoadAverage=252, ProcessCount=168, AvailableMemoryMB=3486 2023-02-10 07:54:28,323 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:28,326 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:54:28,326 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:54:28,327 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:28,329 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:54:28,329 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:54:28,331 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,331 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd empty. 2023-02-10 07:54:28,332 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,332 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-10 07:54:28,342 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:28,343 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => c19064efdfba6693383b2ff7c0db95fd, NAME => 'testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:54:28,350 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:28,350 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing c19064efdfba6693383b2ff7c0db95fd, disabling compactions & flushes 2023-02-10 07:54:28,350 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,350 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,350 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. after waiting 0 ms 2023-02-10 07:54:28,350 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,350 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,350 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for c19064efdfba6693383b2ff7c0db95fd: Waiting for close lock at 1676015668350Disabling compacts and flushes for region at 1676015668350Disabling writes for close at 1676015668350Writing region close event to WAL at 1676015668350Closed at 1676015668350 2023-02-10 07:54:28,352 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:54:28,353 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676015668352"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015668352"}]},"ts":"1676015668352"} 2023-02-10 07:54:28,354 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:54:28,355 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:54:28,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015668355"}]},"ts":"1676015668355"} 2023-02-10 07:54:28,356 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-10 07:54:28,358 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:28,359 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:28,359 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:28,359 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:28,359 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:28,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c19064efdfba6693383b2ff7c0db95fd, ASSIGN}] 2023-02-10 07:54:28,361 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c19064efdfba6693383b2ff7c0db95fd, ASSIGN 2023-02-10 07:54:28,361 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c19064efdfba6693383b2ff7c0db95fd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43211,1676015645706; forceNewPlan=false, retain=false 2023-02-10 07:54:28,512 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:28,513 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=c19064efdfba6693383b2ff7c0db95fd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:28,513 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676015668513"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015668513"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015668513"}]},"ts":"1676015668513"} 2023-02-10 07:54:28,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure c19064efdfba6693383b2ff7c0db95fd, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:54:28,667 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:28,668 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 07:54:28,670 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58634, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 07:54:28,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c19064efdfba6693383b2ff7c0db95fd, NAME => 'testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:28,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:28,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:28,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,677 INFO [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,679 DEBUG [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/cf1 2023-02-10 07:54:28,679 DEBUG [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/cf1 2023-02-10 07:54:28,679 INFO [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c19064efdfba6693383b2ff7c0db95fd columnFamilyName cf1 2023-02-10 07:54:28,680 INFO [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] regionserver.HStore(310): Store=c19064efdfba6693383b2ff7c0db95fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:28,680 INFO [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,681 DEBUG [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/cf2 2023-02-10 07:54:28,681 DEBUG [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/cf2 2023-02-10 07:54:28,682 INFO [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c19064efdfba6693383b2ff7c0db95fd columnFamilyName cf2 2023-02-10 07:54:28,682 INFO [StoreOpener-c19064efdfba6693383b2ff7c0db95fd-1] regionserver.HStore(310): Store=c19064efdfba6693383b2ff7c0db95fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:28,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:28,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:28,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c19064efdfba6693383b2ff7c0db95fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65154776, jitterRate=-0.02911818027496338}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:28,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c19064efdfba6693383b2ff7c0db95fd: Running coprocessor pre-open hook at 1676015668676Writing region info on filesystem at 1676015668676Initializing all the Stores at 1676015668677 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015668677Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015668677Cleaning up temporary data from old regions at 1676015668684 (+7 ms)Cleaning up detritus from prior splits at 1676015668685 (+1 ms)Running coprocessor post-open hooks at 1676015668696 (+11 ms)Region opened successfully at 1676015668696 2023-02-10 07:54:28,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd., pid=55, masterSystemTime=1676015668667 2023-02-10 07:54:28,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=c19064efdfba6693383b2ff7c0db95fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:28,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676015668700"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015668700"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015668700"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015668700"}]},"ts":"1676015668700"} 2023-02-10 07:54:28,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-10 07:54:28,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure c19064efdfba6693383b2ff7c0db95fd, server=jenkins-hbase4.apache.org,43211,1676015645706 in 187 msec 2023-02-10 07:54:28,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-10 07:54:28,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c19064efdfba6693383b2ff7c0db95fd, ASSIGN in 344 msec 2023-02-10 07:54:28,706 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:54:28,706 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015668706"}]},"ts":"1676015668706"} 2023-02-10 07:54:28,707 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-10 07:54:28,709 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:54:28,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 383 msec 2023-02-10 07:54:28,829 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:28,830 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:28,831 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:28,831 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:28,831 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:28,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c19064efdfba6693383b2ff7c0db95fd, UNASSIGN}] 2023-02-10 07:54:28,833 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c19064efdfba6693383b2ff7c0db95fd, UNASSIGN 2023-02-10 07:54:28,834 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=c19064efdfba6693383b2ff7c0db95fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:28,834 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676015668834"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015668834"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015668834"}]},"ts":"1676015668834"} 2023-02-10 07:54:28,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure c19064efdfba6693383b2ff7c0db95fd, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:54:28,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c19064efdfba6693383b2ff7c0db95fd, disabling compactions & flushes 2023-02-10 07:54:28,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. after waiting 0 ms 2023-02-10 07:54:28,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-10 07:54:28,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:28,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd. 2023-02-10 07:54:28,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c19064efdfba6693383b2ff7c0db95fd: Waiting for close lock at 1676015668987Running coprocessor pre-close hooks at 1676015668987Disabling compacts and flushes for region at 1676015668987Disabling writes for close at 1676015668988 (+1 ms)Writing region close event to WAL at 1676015668989 (+1 ms)Running coprocessor post-close hooks at 1676015668992 (+3 ms)Closed at 1676015668993 (+1 ms) 2023-02-10 07:54:28,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:28,995 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=c19064efdfba6693383b2ff7c0db95fd, regionState=CLOSED 2023-02-10 07:54:28,995 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676015668995"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015668995"}]},"ts":"1676015668995"} 2023-02-10 07:54:28,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-10 07:54:28,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure c19064efdfba6693383b2ff7c0db95fd, server=jenkins-hbase4.apache.org,43211,1676015645706 in 161 msec 2023-02-10 07:54:28,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-10 07:54:28,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=c19064efdfba6693383b2ff7c0db95fd, UNASSIGN in 164 msec 2023-02-10 07:54:29,023 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-10 07:54:29,025 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-10 07:54:29,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676015669027"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676015669027"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676015669027"}]},"ts":"1676015669027"} 2023-02-10 07:54:29,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669027"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015669027"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015669027"}]},"ts":"1676015669027"} 2023-02-10 07:54:29,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669027"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015669027"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015669027"}]},"ts":"1676015669027"} 2023-02-10 07:54:29,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ed10e80b06bcb9df7beac19c057004b, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b6f91ff4a65b935935650d5b31c66506, ASSIGN}] 2023-02-10 07:54:29,031 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b6f91ff4a65b935935650d5b31c66506, ASSIGN 2023-02-10 07:54:29,031 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b6f91ff4a65b935935650d5b31c66506, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43211,1676015645706; forceNewPlan=false, retain=false 2023-02-10 07:54:29,032 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ed10e80b06bcb9df7beac19c057004b, ASSIGN 2023-02-10 07:54:29,033 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ed10e80b06bcb9df7beac19c057004b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43211,1676015645706; forceNewPlan=false, retain=false 2023-02-10 07:54:29,182 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-10 07:54:29,183 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=b6f91ff4a65b935935650d5b31c66506, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:29,183 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669183"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015669183"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015669183"}]},"ts":"1676015669183"} 2023-02-10 07:54:29,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure b6f91ff4a65b935935650d5b31c66506, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:54:29,187 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=6ed10e80b06bcb9df7beac19c057004b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:29,188 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669187"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015669187"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015669187"}]},"ts":"1676015669187"} 2023-02-10 07:54:29,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 6ed10e80b06bcb9df7beac19c057004b, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:54:29,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. 2023-02-10 07:54:29,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b6f91ff4a65b935935650d5b31c66506, NAME => 'testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.', STARTKEY => '41', ENDKEY => ''} 2023-02-10 07:54:29,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:29,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:29,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,342 INFO [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,344 DEBUG [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/cf1 2023-02-10 07:54:29,344 DEBUG [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/cf1 2023-02-10 07:54:29,344 INFO [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6f91ff4a65b935935650d5b31c66506 columnFamilyName cf1 2023-02-10 07:54:29,345 INFO [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] regionserver.HStore(310): Store=b6f91ff4a65b935935650d5b31c66506/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:29,345 INFO [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,346 DEBUG [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/cf2 2023-02-10 07:54:29,346 DEBUG [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/cf2 2023-02-10 07:54:29,347 INFO [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6f91ff4a65b935935650d5b31c66506 columnFamilyName cf2 2023-02-10 07:54:29,348 INFO [StoreOpener-b6f91ff4a65b935935650d5b31c66506-1] regionserver.HStore(310): Store=b6f91ff4a65b935935650d5b31c66506/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:29,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:29,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b6f91ff4a65b935935650d5b31c66506; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62400208, jitterRate=-0.07016444206237793}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:29,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b6f91ff4a65b935935650d5b31c66506: Running coprocessor pre-open hook at 1676015669341Writing region info on filesystem at 1676015669341Initializing all the Stores at 1676015669342 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015669342Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015669342Cleaning up temporary data from old regions at 1676015669350 (+8 ms)Cleaning up detritus from prior splits at 1676015669351 (+1 ms)Running coprocessor post-open hooks at 1676015669353 (+2 ms)Region opened successfully at 1676015669353 2023-02-10 07:54:29,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506., pid=61, masterSystemTime=1676015669337 2023-02-10 07:54:29,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. 2023-02-10 07:54:29,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. 2023-02-10 07:54:29,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. 2023-02-10 07:54:29,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6ed10e80b06bcb9df7beac19c057004b, NAME => 'testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.', STARTKEY => '', ENDKEY => '41'} 2023-02-10 07:54:29,356 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=b6f91ff4a65b935935650d5b31c66506, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:29,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:29,356 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669356"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015669356"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015669356"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015669356"}]},"ts":"1676015669356"} 2023-02-10 07:54:29,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:29,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,357 INFO [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-10 07:54:29,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure b6f91ff4a65b935935650d5b31c66506, server=jenkins-hbase4.apache.org,43211,1676015645706 in 172 msec 2023-02-10 07:54:29,359 DEBUG [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/cf1 2023-02-10 07:54:29,359 DEBUG [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/cf1 2023-02-10 07:54:29,359 INFO [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ed10e80b06bcb9df7beac19c057004b columnFamilyName cf1 2023-02-10 07:54:29,360 INFO [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] regionserver.HStore(310): Store=6ed10e80b06bcb9df7beac19c057004b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:29,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b6f91ff4a65b935935650d5b31c66506, ASSIGN in 329 msec 2023-02-10 07:54:29,360 INFO [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,361 DEBUG [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/cf2 2023-02-10 07:54:29,361 DEBUG [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/cf2 2023-02-10 07:54:29,362 INFO [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ed10e80b06bcb9df7beac19c057004b columnFamilyName cf2 2023-02-10 07:54:29,362 INFO [StoreOpener-6ed10e80b06bcb9df7beac19c057004b-1] regionserver.HStore(310): Store=6ed10e80b06bcb9df7beac19c057004b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:29,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:29,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6ed10e80b06bcb9df7beac19c057004b; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66728344, jitterRate=-0.00567018985748291}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:29,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6ed10e80b06bcb9df7beac19c057004b: Running coprocessor pre-open hook at 1676015669356Writing region info on filesystem at 1676015669356Initializing all the Stores at 1676015669357 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015669357Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015669357Cleaning up temporary data from old regions at 1676015669364 (+7 ms)Cleaning up detritus from prior splits at 1676015669364Running coprocessor post-open hooks at 1676015669367 (+3 ms)Region opened successfully at 1676015669367 2023-02-10 07:54:29,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b., pid=62, masterSystemTime=1676015669337 2023-02-10 07:54:29,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. 2023-02-10 07:54:29,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. 2023-02-10 07:54:29,369 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=6ed10e80b06bcb9df7beac19c057004b, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:29,369 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669369"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015669369"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015669369"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015669369"}]},"ts":"1676015669369"} 2023-02-10 07:54:29,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-02-10 07:54:29,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 6ed10e80b06bcb9df7beac19c057004b, server=jenkins-hbase4.apache.org,43211,1676015645706 in 181 msec 2023-02-10 07:54:29,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-10 07:54:29,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ed10e80b06bcb9df7beac19c057004b, ASSIGN in 342 msec 2023-02-10 07:54:29,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 in 544 msec 2023-02-10 07:54:29,585 DEBUG [hconnection-0x7e2f5f25-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-10 07:54:29,589 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58646, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-10 07:54:29,590 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-10 07:54:29,594 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:29,596 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,596 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-10 07:54:29,599 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015669599"}]},"ts":"1676015669599"} 2023-02-10 07:54:29,601 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-10 07:54:29,602 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-10 07:54:29,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ed10e80b06bcb9df7beac19c057004b, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b6f91ff4a65b935935650d5b31c66506, UNASSIGN}] 2023-02-10 07:54:29,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b6f91ff4a65b935935650d5b31c66506, UNASSIGN 2023-02-10 07:54:29,604 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=b6f91ff4a65b935935650d5b31c66506, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:29,605 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669604"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015669604"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015669604"}]},"ts":"1676015669604"} 2023-02-10 07:54:29,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure b6f91ff4a65b935935650d5b31c66506, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:54:29,608 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ed10e80b06bcb9df7beac19c057004b, UNASSIGN 2023-02-10 07:54:29,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=6ed10e80b06bcb9df7beac19c057004b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:29,609 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669609"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015669609"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015669609"}]},"ts":"1676015669609"} 2023-02-10 07:54:29,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 6ed10e80b06bcb9df7beac19c057004b, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:54:29,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-10 07:54:29,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b6f91ff4a65b935935650d5b31c66506, disabling compactions & flushes 2023-02-10 07:54:29,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. 2023-02-10 07:54:29,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. 2023-02-10 07:54:29,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. after waiting 0 ms 2023-02-10 07:54:29,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. 2023-02-10 07:54:29,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-10 07:54:29,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:29,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506. 2023-02-10 07:54:29,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b6f91ff4a65b935935650d5b31c66506: Waiting for close lock at 1676015669758Running coprocessor pre-close hooks at 1676015669758Disabling compacts and flushes for region at 1676015669758Disabling writes for close at 1676015669759 (+1 ms)Writing region close event to WAL at 1676015669759Running coprocessor post-close hooks at 1676015669763 (+4 ms)Closed at 1676015669763 2023-02-10 07:54:29,765 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,765 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6ed10e80b06bcb9df7beac19c057004b, disabling compactions & flushes 2023-02-10 07:54:29,766 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. 2023-02-10 07:54:29,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. 2023-02-10 07:54:29,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. after waiting 0 ms 2023-02-10 07:54:29,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. 2023-02-10 07:54:29,766 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=b6f91ff4a65b935935650d5b31c66506, regionState=CLOSED 2023-02-10 07:54:29,766 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669766"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015669766"}]},"ts":"1676015669766"} 2023-02-10 07:54:29,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-10 07:54:29,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-10 07:54:29,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure b6f91ff4a65b935935650d5b31c66506, server=jenkins-hbase4.apache.org,43211,1676015645706 in 161 msec 2023-02-10 07:54:29,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:29,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b. 2023-02-10 07:54:29,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6ed10e80b06bcb9df7beac19c057004b: Waiting for close lock at 1676015669765Running coprocessor pre-close hooks at 1676015669765Disabling compacts and flushes for region at 1676015669765Disabling writes for close at 1676015669766 (+1 ms)Writing region close event to WAL at 1676015669766Running coprocessor post-close hooks at 1676015669770 (+4 ms)Closed at 1676015669770 2023-02-10 07:54:29,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=b6f91ff4a65b935935650d5b31c66506, UNASSIGN in 167 msec 2023-02-10 07:54:29,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,772 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=6ed10e80b06bcb9df7beac19c057004b, regionState=CLOSED 2023-02-10 07:54:29,772 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015669772"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015669772"}]},"ts":"1676015669772"} 2023-02-10 07:54:29,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-10 07:54:29,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 6ed10e80b06bcb9df7beac19c057004b, server=jenkins-hbase4.apache.org,43211,1676015645706 in 163 msec 2023-02-10 07:54:29,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-10 07:54:29,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ed10e80b06bcb9df7beac19c057004b, UNASSIGN in 172 msec 2023-02-10 07:54:29,777 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015669777"}]},"ts":"1676015669777"} 2023-02-10 07:54:29,778 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-10 07:54:29,780 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-10 07:54:29,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 185 msec 2023-02-10 07:54:29,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-10 07:54:29,902 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-10 07:54:29,902 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-10 07:54:29,911 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:29,911 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,911 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,913 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/recovered.edits] 2023-02-10 07:54:29,913 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/recovered.edits] 2023-02-10 07:54:29,914 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/recovered.edits] 2023-02-10 07:54:29,921 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/recovered.edits/4.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd/recovered.edits/4.seqid 2023-02-10 07:54:29,921 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/recovered.edits/7.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b/recovered.edits/7.seqid 2023-02-10 07:54:29,921 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/recovered.edits/7.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506/recovered.edits/7.seqid 2023-02-10 07:54:29,922 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/c19064efdfba6693383b2ff7c0db95fd 2023-02-10 07:54:29,922 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/6ed10e80b06bcb9df7beac19c057004b 2023-02-10 07:54:29,922 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionNoStoreFile/b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:54:29,922 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-10 07:54:29,924 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,926 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-10 07:54:29,927 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-10 07:54:29,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-10 07:54:29,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015669928"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:29,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015669928"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:29,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015669928"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:29,930 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-10 07:54:29,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => c19064efdfba6693383b2ff7c0db95fd, NAME => 'testSplitTableRegionNoStoreFile,,1676015668326.c19064efdfba6693383b2ff7c0db95fd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6ed10e80b06bcb9df7beac19c057004b, NAME => 'testSplitTableRegionNoStoreFile,,1676015668829.6ed10e80b06bcb9df7beac19c057004b.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b6f91ff4a65b935935650d5b31c66506, NAME => 'testSplitTableRegionNoStoreFile,41,1676015668829.b6f91ff4a65b935935650d5b31c66506.', STARTKEY => '41', ENDKEY => ''}] 2023-02-10 07:54:29,930 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-10 07:54:29,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015669930"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:29,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-10 07:54:29,934 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:54:29,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 32 msec 2023-02-10 07:54:30,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-10 07:54:30,007 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-10 07:54:30,022 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406 (was 397) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_493369716_15 at /127.0.0.1:52532 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_493369716_15 at /127.0.0.1:55696 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=886 (was 882) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=233 (was 252), ProcessCount=168 (was 168), AvailableMemoryMB=3484 (was 3486) 2023-02-10 07:54:30,036 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406, OpenFileDescriptor=886, MaxFileDescriptor=60000, SystemLoadAverage=233, ProcessCount=168, AvailableMemoryMB=3483 2023-02-10 07:54:30,036 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:30,038 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:54:30,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:54:30,040 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,041 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:54:30,042 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:54:30,043 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,044 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f empty. 2023-02-10 07:54:30,044 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,044 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-10 07:54:30,060 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:30,061 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 09f620010a784524f1f7bb2817b44a9f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:54:30,074 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:30,075 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 09f620010a784524f1f7bb2817b44a9f, disabling compactions & flushes 2023-02-10 07:54:30,075 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,075 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,075 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. after waiting 0 ms 2023-02-10 07:54:30,075 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,075 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,075 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 09f620010a784524f1f7bb2817b44a9f: Waiting for close lock at 1676015670075Disabling compacts and flushes for region at 1676015670075Disabling writes for close at 1676015670075Writing region close event to WAL at 1676015670075Closed at 1676015670075 2023-02-10 07:54:30,077 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:54:30,078 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676015670078"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015670078"}]},"ts":"1676015670078"} 2023-02-10 07:54:30,080 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:54:30,081 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:54:30,081 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015670081"}]},"ts":"1676015670081"} 2023-02-10 07:54:30,082 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-10 07:54:30,084 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:30,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:30,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:30,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:30,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:30,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=09f620010a784524f1f7bb2817b44a9f, ASSIGN}] 2023-02-10 07:54:30,087 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=09f620010a784524f1f7bb2817b44a9f, ASSIGN 2023-02-10 07:54:30,088 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=09f620010a784524f1f7bb2817b44a9f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43281,1676015645662; forceNewPlan=false, retain=false 2023-02-10 07:54:30,238 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:30,239 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=09f620010a784524f1f7bb2817b44a9f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676015670239"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015670239"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015670239"}]},"ts":"1676015670239"} 2023-02-10 07:54:30,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 09f620010a784524f1f7bb2817b44a9f, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:30,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 09f620010a784524f1f7bb2817b44a9f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:30,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:30,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:30,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,397 INFO [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,399 DEBUG [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1 2023-02-10 07:54:30,399 DEBUG [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1 2023-02-10 07:54:30,399 INFO [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09f620010a784524f1f7bb2817b44a9f columnFamilyName cf1 2023-02-10 07:54:30,399 INFO [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] regionserver.HStore(310): Store=09f620010a784524f1f7bb2817b44a9f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:30,400 INFO [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,401 DEBUG [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2 2023-02-10 07:54:30,401 DEBUG [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2 2023-02-10 07:54:30,401 INFO [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09f620010a784524f1f7bb2817b44a9f columnFamilyName cf2 2023-02-10 07:54:30,402 INFO [StoreOpener-09f620010a784524f1f7bb2817b44a9f-1] regionserver.HStore(310): Store=09f620010a784524f1f7bb2817b44a9f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:30,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:30,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:30,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:30,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 09f620010a784524f1f7bb2817b44a9f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70332388, jitterRate=0.04803425073623657}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:30,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 09f620010a784524f1f7bb2817b44a9f: Running coprocessor pre-open hook at 1676015670396Writing region info on filesystem at 1676015670396Initializing all the Stores at 1676015670397 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015670397Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015670397Cleaning up temporary data from old regions at 1676015670404 (+7 ms)Cleaning up detritus from prior splits at 1676015670404Running coprocessor post-open hooks at 1676015670410 (+6 ms)Region opened successfully at 1676015670410 2023-02-10 07:54:30,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f., pid=71, masterSystemTime=1676015670393 2023-02-10 07:54:30,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=09f620010a784524f1f7bb2817b44a9f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676015670413"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015670413"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015670413"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015670413"}]},"ts":"1676015670413"} 2023-02-10 07:54:30,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-10 07:54:30,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 09f620010a784524f1f7bb2817b44a9f, server=jenkins-hbase4.apache.org,43281,1676015645662 in 174 msec 2023-02-10 07:54:30,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-10 07:54:30,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=09f620010a784524f1f7bb2817b44a9f, ASSIGN in 331 msec 2023-02-10 07:54:30,421 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:54:30,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015670421"}]},"ts":"1676015670421"} 2023-02-10 07:54:30,422 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-10 07:54:30,425 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:54:30,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 386 msec 2023-02-10 07:54:30,548 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:30,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,550 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:30,550 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,550 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:30,550 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:30,550 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,550 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,551 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,551 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,551 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,551 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,552 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,552 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,553 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,553 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:30,553 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,554 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:54:30,554 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,554 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,556 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,571 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/404ccfb3ca4741e8a71deb9f03198342 2023-02-10 07:54:30,589 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/8d7f9da32de149ceb43b4f6354a06404 2023-02-10 07:54:30,594 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/404ccfb3ca4741e8a71deb9f03198342 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342 2023-02-10 07:54:30,598 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:30,599 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/8d7f9da32de149ceb43b4f6354a06404 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404 2023-02-10 07:54:30,603 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:54:30,604 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 50ms, sequenceid=6, compaction requested=false 2023-02-10 07:54:30,604 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:30,604 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:30,605 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,605 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:30,605 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:30,605 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,605 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,605 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,607 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,607 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,608 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,608 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:30,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c6a5043[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:30,610 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:30,610 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,611 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,611 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:30,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,611 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,611 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,611 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:30,611 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:30,611 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,612 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,612 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:30,613 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,613 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,613 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,613 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,613 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,614 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,613 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,614 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,614 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,614 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:30,617 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:30,617 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:30,617 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:30,617 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:30,617 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,617 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:30,620 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,620 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,620 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,621 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,621 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,621 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,622 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,623 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,624 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,624 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,631 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,631 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:30,631 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,632 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:30,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,632 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:30,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,732 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:30,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:30,748 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:30,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:30,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,750 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:30,750 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,750 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:30,750 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:30,750 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,750 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,752 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,752 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,752 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,752 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,752 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,752 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,753 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:30,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:30,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:30,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:30,754 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,754 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:30,755 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:30,755 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:30,757 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,758 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,758 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,758 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,758 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,758 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,766 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/a7971f39a5bf4765aa12f17df2550926 2023-02-10 07:54:30,778 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/3428adab207e43daa350eb819529c3af 2023-02-10 07:54:30,782 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/a7971f39a5bf4765aa12f17df2550926 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926 2023-02-10 07:54:30,786 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:30,787 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/3428adab207e43daa350eb819529c3af as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af 2023-02-10 07:54:30,790 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:54:30,791 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 36ms, sequenceid=19, compaction requested=false 2023-02-10 07:54:30,791 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:30,791 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,791 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:30,791 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:30,791 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,791 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,791 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,793 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,793 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,793 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,793 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:30,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e12e1c7[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:30,795 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:30,795 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,796 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,796 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,796 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,797 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,797 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,796 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:30,796 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:30,797 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:30,797 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,797 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:30,799 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,799 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,799 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,800 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:30,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:30,803 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:30,803 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:30,804 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:30,804 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,804 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:30,805 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,805 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,805 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,805 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,806 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,806 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,807 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,808 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,808 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,809 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,811 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:30,811 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,811 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:30,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,811 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,812 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,813 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,813 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,912 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:30,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:30,927 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:30,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:30,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,928 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:30,929 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,929 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:30,929 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:30,929 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,929 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,932 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,932 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,932 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,932 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,932 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,932 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:30,933 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,933 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:30,933 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,933 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:30,934 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:30,934 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,934 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,934 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,934 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:30,935 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,936 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,946 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/d17d3670df374a748707bce015d1cda8 2023-02-10 07:54:30,961 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/aa6387c3b0e0496d9c762ad9b5f9b216 2023-02-10 07:54:30,966 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/d17d3670df374a748707bce015d1cda8 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8 2023-02-10 07:54:30,970 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:30,971 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/aa6387c3b0e0496d9c762ad9b5f9b216 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216 2023-02-10 07:54:30,974 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:54:30,975 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 41ms, sequenceid=32, compaction requested=true 2023-02-10 07:54:30,975 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:30,975 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:30,976 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:30,976 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:30,976 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,976 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:30,976 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:30,981 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,981 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,982 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,982 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:30,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c5c1739[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:30,984 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:30,984 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,985 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:30,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:30,985 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:30,987 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:30,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,987 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:30,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:30,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:30,991 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:30,991 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:30,991 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:30,991 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,991 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:30,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:30,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,995 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:30,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:30,995 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,995 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:30,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,997 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,997 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:30,998 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:30,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:30,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:30,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:30,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:31,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,000 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:31,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,000 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:31,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,001 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,002 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,002 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,101 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:31,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:31,116 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:31,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:31,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,117 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,117 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,117 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:31,117 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:31,118 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,118 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,119 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,119 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,119 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,119 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,119 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,119 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,121 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,121 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,122 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:31,122 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:31,123 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,123 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,123 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,123 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,123 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,123 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,133 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/6111193081ef4fdcb3a9c353d4b0ea58 2023-02-10 07:54:31,144 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/35ae71e6675241e2b1cca138df49975b 2023-02-10 07:54:31,148 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/6111193081ef4fdcb3a9c353d4b0ea58 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/6111193081ef4fdcb3a9c353d4b0ea58 2023-02-10 07:54:31,151 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/6111193081ef4fdcb3a9c353d4b0ea58, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:31,152 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/35ae71e6675241e2b1cca138df49975b as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/35ae71e6675241e2b1cca138df49975b 2023-02-10 07:54:31,156 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/35ae71e6675241e2b1cca138df49975b, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:54:31,156 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 34ms, sequenceid=45, compaction requested=true 2023-02-10 07:54:31,157 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:31,157 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,157 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:31,157 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:31,157 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,157 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,157 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,159 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,159 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,159 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,159 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:31,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63ed17c1[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:31,161 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:31,161 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,162 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,162 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,162 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,162 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,162 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:31,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,163 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:31,164 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,164 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,164 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,164 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,165 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,165 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,165 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,165 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,165 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,165 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:31,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:31,168 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:31,168 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:31,168 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:31,168 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,168 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:31,172 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,172 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,172 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,172 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,172 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,173 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,173 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,174 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,174 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,176 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,176 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,176 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,179 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,178 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:31,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,179 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,279 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:31,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:31,292 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:31,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:31,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,294 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,294 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,294 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:31,294 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:31,294 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,294 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,295 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,295 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,295 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,295 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,295 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,296 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,297 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,297 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:31,297 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:31,298 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,298 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:31,298 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:31,299 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,300 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,300 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,300 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,300 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,300 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,308 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/1c0466ba52da4059bf3208f176c434df 2023-02-10 07:54:31,322 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/8b31ef218cf4459fa3297d31bf9fa8d6 2023-02-10 07:54:31,327 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/1c0466ba52da4059bf3208f176c434df as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/1c0466ba52da4059bf3208f176c434df 2023-02-10 07:54:31,332 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/1c0466ba52da4059bf3208f176c434df, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:31,333 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/8b31ef218cf4459fa3297d31bf9fa8d6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8b31ef218cf4459fa3297d31bf9fa8d6 2023-02-10 07:54:31,337 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8b31ef218cf4459fa3297d31bf9fa8d6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:54:31,338 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 40ms, sequenceid=58, compaction requested=true 2023-02-10 07:54:31,338 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:31,338 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,338 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:31,338 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:31,338 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,338 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,338 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,340 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,340 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,340 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,340 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:31,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d3babb7[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:31,342 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:31,343 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,344 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:31,344 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,344 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,344 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:31,344 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,344 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,344 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,345 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,345 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,345 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:31,346 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,346 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,346 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,347 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,347 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,347 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,347 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,347 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,348 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:31,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:31,351 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:31,351 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:31,351 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:31,351 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,351 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:31,353 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,353 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,353 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,353 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,354 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,354 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,356 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,357 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,357 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,360 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,360 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,360 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:31,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,360 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,361 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,461 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:31,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:31,474 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:31,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:31,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,475 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,475 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,475 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:31,475 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:31,476 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,476 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,480 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,480 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,480 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,480 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,481 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,481 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,482 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,482 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:31,483 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,483 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:31,483 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:54:31,484 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,485 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,485 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,485 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,485 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,485 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,492 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/744ea1a98ace46eb98c8596a9d6b204f 2023-02-10 07:54:31,504 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/1744974f9a2a41aba80883aa57b3fdce 2023-02-10 07:54:31,508 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/744ea1a98ace46eb98c8596a9d6b204f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/744ea1a98ace46eb98c8596a9d6b204f 2023-02-10 07:54:31,511 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/744ea1a98ace46eb98c8596a9d6b204f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:31,512 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/1744974f9a2a41aba80883aa57b3fdce as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/1744974f9a2a41aba80883aa57b3fdce 2023-02-10 07:54:31,515 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/1744974f9a2a41aba80883aa57b3fdce, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:54:31,516 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 33ms, sequenceid=71, compaction requested=true 2023-02-10 07:54:31,516 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:31,516 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,516 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:31,516 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:31,516 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,516 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,516 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,518 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,518 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,518 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:31,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@443e750e[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:31,520 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:31,520 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,522 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,522 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:31,522 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,522 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:31,522 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,523 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:31,524 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,524 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,524 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,524 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,525 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,525 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,525 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,525 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,525 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,525 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:31,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:31,527 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:31,527 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:31,527 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:31,527 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,527 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:31,529 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,529 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,529 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,529 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,529 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,530 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,530 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,531 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,532 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,532 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,535 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,535 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,535 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,536 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,536 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,536 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:31,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,535 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,536 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,536 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,536 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,536 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,636 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:31,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:31,650 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:31,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:31,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,651 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,651 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,651 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:31,651 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:31,652 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,652 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,653 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,653 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,653 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:31,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,655 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,655 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:31,656 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:31,656 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,656 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:31,656 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-10 07:54:31,656 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,657 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,657 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,657 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,657 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,657 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,665 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/e23ea22ec97f47c3afe664026051c49c 2023-02-10 07:54:31,668 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e23ea22ec97f47c3afe664026051c49c 2023-02-10 07:54:31,676 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/406b8e1f39f645188641a4fdd8c8bb24 2023-02-10 07:54:31,679 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 406b8e1f39f645188641a4fdd8c8bb24 2023-02-10 07:54:31,680 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/e23ea22ec97f47c3afe664026051c49c as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c 2023-02-10 07:54:31,683 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e23ea22ec97f47c3afe664026051c49c 2023-02-10 07:54:31,683 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c, entries=9, sequenceid=83, filesize=5.1 K 2023-02-10 07:54:31,684 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/406b8e1f39f645188641a4fdd8c8bb24 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24 2023-02-10 07:54:31,687 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 406b8e1f39f645188641a4fdd8c8bb24 2023-02-10 07:54:31,687 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24, entries=9, sequenceid=83, filesize=5.1 K 2023-02-10 07:54:31,688 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 32ms, sequenceid=83, compaction requested=true 2023-02-10 07:54:31,688 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:31,688 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,690 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,690 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,690 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,690 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:31,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78e7576f[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:31,692 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:31,692 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,694 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,694 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,694 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,695 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:31,694 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,694 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:31,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,695 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:31,695 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,695 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,695 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:31,700 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,700 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,700 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,700 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,700 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,700 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,700 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,701 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,701 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,701 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:31,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:31,703 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:31,703 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:31,703 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:31,703 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,703 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:31,705 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,705 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,705 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,705 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,705 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,706 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,707 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,708 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,709 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,709 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,711 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,712 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:31,711 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,712 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:31,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,711 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,712 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:31,712 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,712 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,712 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,812 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:31,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:31,821 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:31,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:31,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,823 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,823 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,823 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:31,823 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:31,823 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,823 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,825 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,825 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,825 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,825 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,825 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,825 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,826 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,826 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,827 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,827 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:31,827 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:31,827 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-10 07:54:31,828 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,828 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,829 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,829 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,829 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,829 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,836 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:31,839 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:31,847 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:31,850 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:31,850 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/95013f8f2be74bb5a707cf27d4c41d8e as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:31,854 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:31,854 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/95013f8f2be74bb5a707cf27d4c41d8e, entries=5, sequenceid=91, filesize=5.0 K 2023-02-10 07:54:31,854 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/c0d9c8c781dd4429837b1042675de285 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:31,858 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:31,858 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/c0d9c8c781dd4429837b1042675de285, entries=5, sequenceid=91, filesize=5.0 K 2023-02-10 07:54:31,858 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 31ms, sequenceid=91, compaction requested=true 2023-02-10 07:54:31,858 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:31,859 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,859 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:31,859 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:31,859 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,859 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,859 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,862 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,862 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,862 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,862 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:31,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@733f04ad[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:31,864 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:31,864 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,865 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:31,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:31,865 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:31,866 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,866 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,866 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,866 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:31,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:31,870 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:31,870 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:31,870 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:31,870 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,870 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:31,873 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,873 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,873 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:31,873 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:31,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,874 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,874 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,875 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:31,876 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,876 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,882 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:31,882 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,882 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:31,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:31,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:31,882 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:31,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,883 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,983 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:31,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:31,991 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:54:31,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:54:31,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,993 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:31,993 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,993 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-10 07:54:31,993 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-10 07:54:31,993 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,993 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,994 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,994 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:31,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:31,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,995 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,996 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:31,996 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:31,998 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-10 07:54:31,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,998 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-10 07:54:31,998 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:54:31,998 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,998 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-10 07:54:31,998 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,998 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:54:31,998 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:54:31,998 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:31,999 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. started... 2023-02-10 07:54:31,999 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,999 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,999 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:31,999 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:31,999 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-10 07:54:32,007 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,011 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,018 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:32,021 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:32,022 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/256d2d76498749fab5f4a1bffd7af7be as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,025 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,026 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/256d2d76498749fab5f4a1bffd7af7be, entries=5, sequenceid=99, filesize=5.0 K 2023-02-10 07:54:32,026 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/bc6fb538ab704ec4892c06a0d1419890 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:32,030 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:32,030 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/bc6fb538ab704ec4892c06a0d1419890, entries=5, sequenceid=99, filesize=5.0 K 2023-02-10 07:54:32,030 INFO [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 31ms, sequenceid=99, compaction requested=true 2023-02-10 07:54:32,030 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09f620010a784524f1f7bb2817b44a9f: 2023-02-10 07:54:32,030 DEBUG [rs(jenkins-hbase4.apache.org,43281,1676015645662)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:32,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:54:32,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:54:32,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-10 07:54:32,031 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-10 07:54:32,032 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,032 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:32,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:32,032 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,032 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:54:32,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:32,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-10 07:54:32,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73597f6e[Count = 0] remaining members to acquire global barrier 2023-02-10 07:54:32,034 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-10 07:54:32,034 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:32,035 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,035 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:54:32,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-10 07:54:32,035 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:54:32,037 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:32,037 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,037 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:32,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:32,037 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:32,037 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:32,037 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:32,037 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:32,038 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-10 07:54:32,038 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:32,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:32,038 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-10 07:54:32,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:32,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,43281,1676015645662': 2023-02-10 07:54:32,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:54:32,041 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-10 07:54:32,041 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:54:32,041 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:54:32,041 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,041 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:54:32,042 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:54:32,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:54:32,042 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:32,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:32,042 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:32,043 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,043 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:32,043 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:32,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,043 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:54:32,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,045 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,045 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:54:32,045 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:32,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:32,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:54:32,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:32,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:54:32,047 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:32,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:32,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,048 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-10 07:54:32,048 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:32,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:54:32,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:54:32,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:54:32,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:32,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:32,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:54:32,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,048 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:54:32,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:54:32,049 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:32,049 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,049 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:32,149 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:54:32,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:54:32,152 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,152 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,154 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,154 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,154 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=09f620010a784524f1f7bb2817b44a9f, UNASSIGN}] 2023-02-10 07:54:32,156 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=09f620010a784524f1f7bb2817b44a9f, UNASSIGN 2023-02-10 07:54:32,157 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=09f620010a784524f1f7bb2817b44a9f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,157 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676015672157"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015672157"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015672157"}]},"ts":"1676015672157"} 2023-02-10 07:54:32,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 09f620010a784524f1f7bb2817b44a9f, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:32,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 09f620010a784524f1f7bb2817b44a9f, disabling compactions & flushes 2023-02-10 07:54:32,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:32,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:32,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. after waiting 0 ms 2023-02-10 07:54:32,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:32,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 09f620010a784524f1f7bb2817b44a9f 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-10 07:54:32,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:32,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:32,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf1/8366be06fb074a05aa9774621b31a305 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,347 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/8366be06fb074a05aa9774621b31a305, entries=1, sequenceid=103, filesize=4.9 K 2023-02-10 07:54:32,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.tmp/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:32,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:32,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8, entries=1, sequenceid=103, filesize=4.9 K 2023-02-10 07:54:32,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 41ms, sequenceid=103, compaction requested=true 2023-02-10 07:54:32,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-10 07:54:32,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:32,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. 2023-02-10 07:54:32,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 09f620010a784524f1f7bb2817b44a9f: Waiting for close lock at 1676015672310Running coprocessor pre-close hooks at 1676015672310Disabling compacts and flushes for region at 1676015672310Disabling writes for close at 1676015672310Obtaining lock to block concurrent updates at 1676015672311 (+1 ms)Preparing flush snapshotting stores in 09f620010a784524f1f7bb2817b44a9f at 1676015672311Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676015672311Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f. at 1676015672311Flushing 09f620010a784524f1f7bb2817b44a9f/cf1: creating writer at 1676015672312 (+1 ms)Flushing 09f620010a784524f1f7bb2817b44a9f/cf1: appending metadata at 1676015672314 (+2 ms)Flushing 09f620010a784524f1f7bb2817b44a9f/cf1: closing flushed file at 1676015672314Flushing 09f620010a784524f1f7bb2817b44a9f/cf2: creating writer at 1676015672324 (+10 ms)Flushing 09f620010a784524f1f7bb2817b44a9f/cf2: appending metadata at 1676015672326 (+2 ms)Flushing 09f620010a784524f1f7bb2817b44a9f/cf2: closing flushed file at 1676015672326Flushing 09f620010a784524f1f7bb2817b44a9f/cf1: reopening flushed file at 1676015672343 (+17 ms)Flushing 09f620010a784524f1f7bb2817b44a9f/cf2: reopening flushed file at 1676015672348 (+5 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 09f620010a784524f1f7bb2817b44a9f in 41ms, sequenceid=103, compaction requested=true at 1676015672352 (+4 ms)Writing region close event to WAL at 1676015672354 (+2 ms)Running coprocessor post-close hooks at 1676015672357 (+3 ms)Closed at 1676015672357 2023-02-10 07:54:32,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,359 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=09f620010a784524f1f7bb2817b44a9f, regionState=CLOSED 2023-02-10 07:54:32,359 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676015672359"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015672359"}]},"ts":"1676015672359"} 2023-02-10 07:54:32,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-10 07:54:32,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 09f620010a784524f1f7bb2817b44a9f, server=jenkins-hbase4.apache.org,43281,1676015645662 in 202 msec 2023-02-10 07:54:32,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-10 07:54:32,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=09f620010a784524f1f7bb2817b44a9f, UNASSIGN in 206 msec 2023-02-10 07:54:32,374 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=09f620010a784524f1f7bb2817b44a9f, threads=16 2023-02-10 07:54:32,375 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,376 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/1744974f9a2a41aba80883aa57b3fdce for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,376 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,376 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/35ae71e6675241e2b1cca138df49975b for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,376 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,377 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8b31ef218cf4459fa3297d31bf9fa8d6 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,377 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,377 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,377 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/bc6fb538ab704ec4892c06a0d1419890 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,377 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/c0d9c8c781dd4429837b1042675de285 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,377 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/1c0466ba52da4059bf3208f176c434df for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,377 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/256d2d76498749fab5f4a1bffd7af7be for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,378 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,384 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/6111193081ef4fdcb3a9c353d4b0ea58 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,384 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/744ea1a98ace46eb98c8596a9d6b204f for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,384 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/8366be06fb074a05aa9774621b31a305 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,387 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:32,388 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 406b8e1f39f645188641a4fdd8c8bb24 2023-02-10 07:54:32,391 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:32,391 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:32,395 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/1744974f9a2a41aba80883aa57b3fdce, top=false 2023-02-10 07:54:32,395 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8b31ef218cf4459fa3297d31bf9fa8d6, top=false 2023-02-10 07:54:32,395 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/6111193081ef4fdcb3a9c353d4b0ea58, top=false 2023-02-10 07:54:32,396 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/744ea1a98ace46eb98c8596a9d6b204f, top=false 2023-02-10 07:54:32,397 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/35ae71e6675241e2b1cca138df49975b, top=false 2023-02-10 07:54:32,397 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,399 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,400 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:32,400 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8, top=true 2023-02-10 07:54:32,403 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:32,403 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/c0d9c8c781dd4429837b1042675de285, top=true 2023-02-10 07:54:32,404 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/1c0466ba52da4059bf3208f176c434df, top=false 2023-02-10 07:54:32,404 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:32,404 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/bc6fb538ab704ec4892c06a0d1419890, top=true 2023-02-10 07:54:32,406 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,406 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/8366be06fb074a05aa9774621b31a305, top=true 2023-02-10 07:54:32,407 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,407 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/256d2d76498749fab5f4a1bffd7af7be, top=true 2023-02-10 07:54:32,420 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-0bb230ae00c3463b8a7fdcb4c304dfd8 for child: ecec840173cb92d09b34c16fdbcc87bb, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,420 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,420 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/95013f8f2be74bb5a707cf27d4c41d8e for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,420 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-256d2d76498749fab5f4a1bffd7af7be for child: ecec840173cb92d09b34c16fdbcc87bb, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,420 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/256d2d76498749fab5f4a1bffd7af7be for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,420 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,420 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1744974f9a2a41aba80883aa57b3fdce for child: e55fa0b419778bd958cda69bfff47ba8, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,421 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-bc6fb538ab704ec4892c06a0d1419890 for child: ecec840173cb92d09b34c16fdbcc87bb, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,422 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/bc6fb538ab704ec4892c06a0d1419890 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,422 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,422 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-744ea1a98ace46eb98c8596a9d6b204f for child: e55fa0b419778bd958cda69bfff47ba8, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,422 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1c0466ba52da4059bf3208f176c434df for child: e55fa0b419778bd958cda69bfff47ba8, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,423 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8366be06fb074a05aa9774621b31a305 for child: ecec840173cb92d09b34c16fdbcc87bb, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,423 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/8366be06fb074a05aa9774621b31a305 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,423 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,424 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-35ae71e6675241e2b1cca138df49975b for child: e55fa0b419778bd958cda69bfff47ba8, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,424 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-6111193081ef4fdcb3a9c353d4b0ea58 for child: e55fa0b419778bd958cda69bfff47ba8, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,425 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:32,425 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-c0d9c8c781dd4429837b1042675de285 for child: ecec840173cb92d09b34c16fdbcc87bb, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,425 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/c0d9c8c781dd4429837b1042675de285 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,426 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8b31ef218cf4459fa3297d31bf9fa8d6 for child: e55fa0b419778bd958cda69bfff47ba8, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,427 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/1744974f9a2a41aba80883aa57b3fdce for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,429 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/1c0466ba52da4059bf3208f176c434df for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,430 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/35ae71e6675241e2b1cca138df49975b for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,431 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/744ea1a98ace46eb98c8596a9d6b204f for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,432 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 406b8e1f39f645188641a4fdd8c8bb24 2023-02-10 07:54:32,435 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e23ea22ec97f47c3afe664026051c49c 2023-02-10 07:54:32,436 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/6111193081ef4fdcb3a9c353d4b0ea58 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,436 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:32,437 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/95013f8f2be74bb5a707cf27d4c41d8e, top=true 2023-02-10 07:54:32,441 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8b31ef218cf4459fa3297d31bf9fa8d6 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,452 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,480 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/.splits/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-95013f8f2be74bb5a707cf27d4c41d8e for child: ecec840173cb92d09b34c16fdbcc87bb, parent: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,480 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/95013f8f2be74bb5a707cf27d4c41d8e for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,480 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,482 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,488 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,488 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,492 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e23ea22ec97f47c3afe664026051c49c 2023-02-10 07:54:32,496 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,504 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8 for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,507 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c for region: 09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,508 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 09f620010a784524f1f7bb2817b44a9f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:54:32,545 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-10 07:54:32,546 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-10 07:54:32,548 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676015672548"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676015672548"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676015672548"}]},"ts":"1676015672548"} 2023-02-10 07:54:32,548 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015672548"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015672548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015672548"}]},"ts":"1676015672548"} 2023-02-10 07:54:32,548 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015672548"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015672548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015672548"}]},"ts":"1676015672548"} 2023-02-10 07:54:32,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e55fa0b419778bd958cda69bfff47ba8, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ecec840173cb92d09b34c16fdbcc87bb, ASSIGN}] 2023-02-10 07:54:32,552 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ecec840173cb92d09b34c16fdbcc87bb, ASSIGN 2023-02-10 07:54:32,552 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ecec840173cb92d09b34c16fdbcc87bb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43281,1676015645662; forceNewPlan=false, retain=false 2023-02-10 07:54:32,553 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e55fa0b419778bd958cda69bfff47ba8, ASSIGN 2023-02-10 07:54:32,554 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e55fa0b419778bd958cda69bfff47ba8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43281,1676015645662; forceNewPlan=false, retain=false 2023-02-10 07:54:32,703 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-10 07:54:32,704 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=ecec840173cb92d09b34c16fdbcc87bb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,704 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015672704"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015672704"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015672704"}]},"ts":"1676015672704"} 2023-02-10 07:54:32,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure ecec840173cb92d09b34c16fdbcc87bb, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:32,708 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=e55fa0b419778bd958cda69bfff47ba8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,708 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015672708"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015672708"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015672708"}]},"ts":"1676015672708"} 2023-02-10 07:54:32,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure e55fa0b419778bd958cda69bfff47ba8, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:32,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:32,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e55fa0b419778bd958cda69bfff47ba8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.', STARTKEY => '', ENDKEY => '60'} 2023-02-10 07:54:32,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:32,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:32,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,862 INFO [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,863 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1 2023-02-10 07:54:32,863 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1 2023-02-10 07:54:32,863 INFO [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e55fa0b419778bd958cda69bfff47ba8 columnFamilyName cf1 2023-02-10 07:54:32,874 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342-bottom 2023-02-10 07:54:32,878 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926-bottom 2023-02-10 07:54:32,882 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8-bottom 2023-02-10 07:54:32,885 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,886 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c-bottom 2023-02-10 07:54:32,889 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1c0466ba52da4059bf3208f176c434df 2023-02-10 07:54:32,892 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-6111193081ef4fdcb3a9c353d4b0ea58 2023-02-10 07:54:32,896 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-744ea1a98ace46eb98c8596a9d6b204f 2023-02-10 07:54:32,896 INFO [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(310): Store=e55fa0b419778bd958cda69bfff47ba8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:32,896 INFO [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,897 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2 2023-02-10 07:54:32,897 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2 2023-02-10 07:54:32,898 INFO [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e55fa0b419778bd958cda69bfff47ba8 columnFamilyName cf2 2023-02-10 07:54:32,909 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af-bottom 2023-02-10 07:54:32,912 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,914 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24-bottom 2023-02-10 07:54:32,919 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404-bottom 2023-02-10 07:54:32,923 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216-bottom 2023-02-10 07:54:32,926 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1744974f9a2a41aba80883aa57b3fdce 2023-02-10 07:54:32,928 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-35ae71e6675241e2b1cca138df49975b 2023-02-10 07:54:32,932 DEBUG [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8b31ef218cf4459fa3297d31bf9fa8d6 2023-02-10 07:54:32,932 INFO [StoreOpener-e55fa0b419778bd958cda69bfff47ba8-1] regionserver.HStore(310): Store=e55fa0b419778bd958cda69bfff47ba8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:32,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:32,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:32,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e55fa0b419778bd958cda69bfff47ba8; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70181517, jitterRate=0.04578609764575958}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:32,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e55fa0b419778bd958cda69bfff47ba8: Running coprocessor pre-open hook at 1676015672861Writing region info on filesystem at 1676015672861Initializing all the Stores at 1676015672862 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015672862Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015672862Cleaning up temporary data from old regions at 1676015672934 (+72 ms)Cleaning up detritus from prior splits at 1676015672935 (+1 ms)Running coprocessor post-open hooks at 1676015672937 (+2 ms)Region opened successfully at 1676015672937 2023-02-10 07:54:32,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8., pid=78, masterSystemTime=1676015672857 2023-02-10 07:54:32,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:32,940 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:32,940 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:32,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:32,940 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:32,940 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:32,940 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1912): e55fa0b419778bd958cda69bfff47ba8/cf1 is initiating minor compaction (all files) 2023-02-10 07:54:32,940 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1912): e55fa0b419778bd958cda69bfff47ba8/cf2 is initiating minor compaction (all files) 2023-02-10 07:54:32,940 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e55fa0b419778bd958cda69bfff47ba8/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:32,940 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e55fa0b419778bd958cda69bfff47ba8/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:32,940 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-35ae71e6675241e2b1cca138df49975b, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8b31ef218cf4459fa3297d31bf9fa8d6, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1744974f9a2a41aba80883aa57b3fdce, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp, totalSize=35.0 K 2023-02-10 07:54:32,940 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-6111193081ef4fdcb3a9c353d4b0ea58, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1c0466ba52da4059bf3208f176c434df, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-744ea1a98ace46eb98c8596a9d6b204f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp, totalSize=35.0 K 2023-02-10 07:54:32,941 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015670543 2023-02-10 07:54:32,941 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting 404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015670543 2023-02-10 07:54:32,941 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015670734 2023-02-10 07:54:32,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:32,942 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015670734 2023-02-10 07:54:32,942 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015670914 2023-02-10 07:54:32,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:32,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:32,942 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=e55fa0b419778bd958cda69bfff47ba8, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:32,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ecec840173cb92d09b34c16fdbcc87bb, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.', STARTKEY => '60', ENDKEY => ''} 2023-02-10 07:54:32,942 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015672942"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015672942"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015672942"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015672942"}]},"ts":"1676015672942"} 2023-02-10 07:54:32,943 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015670914 2023-02-10 07:54:32,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:32,943 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-35ae71e6675241e2b1cca138df49975b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015671103 2023-02-10 07:54:32,943 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-6111193081ef4fdcb3a9c353d4b0ea58, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015671103 2023-02-10 07:54:32,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:32,944 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8b31ef218cf4459fa3297d31bf9fa8d6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015671281 2023-02-10 07:54:32,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,944 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1c0466ba52da4059bf3208f176c434df, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015671281 2023-02-10 07:54:32,944 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1744974f9a2a41aba80883aa57b3fdce, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015671463 2023-02-10 07:54:32,944 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015671638 2023-02-10 07:54:32,945 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-744ea1a98ace46eb98c8596a9d6b204f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015671463 2023-02-10 07:54:32,945 INFO [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,945 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015671638 2023-02-10 07:54:32,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-10 07:54:32,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure e55fa0b419778bd958cda69bfff47ba8, server=jenkins-hbase4.apache.org,43281,1676015645662 in 235 msec 2023-02-10 07:54:32,947 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1 2023-02-10 07:54:32,948 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1 2023-02-10 07:54:32,948 INFO [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ecec840173cb92d09b34c16fdbcc87bb columnFamilyName cf1 2023-02-10 07:54:32,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e55fa0b419778bd958cda69bfff47ba8, ASSIGN in 396 msec 2023-02-10 07:54:32,960 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342-top 2023-02-10 07:54:32,964 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] throttle.PressureAwareThroughputController(145): e55fa0b419778bd958cda69bfff47ba8#cf2#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:32,964 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] throttle.PressureAwareThroughputController(145): e55fa0b419778bd958cda69bfff47ba8#cf1#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:32,966 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926-top 2023-02-10 07:54:32,969 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8-top 2023-02-10 07:54:32,975 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:32,975 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c-top 2023-02-10 07:54:32,979 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,979 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:32,983 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,983 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:32,985 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp/cf1/4b71e1b1a51c48d4bddac1d8303ad10d as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/4b71e1b1a51c48d4bddac1d8303ad10d 2023-02-10 07:54:32,987 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp/cf2/ad5814dc2e3242abb9ee3824b935b49f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/ad5814dc2e3242abb9ee3824b935b49f 2023-02-10 07:54:32,987 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:32,987 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:32,988 INFO [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(310): Store=ecec840173cb92d09b34c16fdbcc87bb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:32,988 INFO [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:32,989 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2 2023-02-10 07:54:32,989 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2 2023-02-10 07:54:32,989 INFO [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ecec840173cb92d09b34c16fdbcc87bb columnFamilyName cf2 2023-02-10 07:54:32,990 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e55fa0b419778bd958cda69bfff47ba8/cf1 of e55fa0b419778bd958cda69bfff47ba8 into 4b71e1b1a51c48d4bddac1d8303ad10d(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:32,990 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e55fa0b419778bd958cda69bfff47ba8: 2023-02-10 07:54:32,990 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8., storeName=e55fa0b419778bd958cda69bfff47ba8/cf1, priority=9, startTime=1676015672939; duration=0sec 2023-02-10 07:54:32,990 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:32,993 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e55fa0b419778bd958cda69bfff47ba8/cf2 of e55fa0b419778bd958cda69bfff47ba8 into ad5814dc2e3242abb9ee3824b935b49f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:32,993 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e55fa0b419778bd958cda69bfff47ba8: 2023-02-10 07:54:32,993 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8., storeName=e55fa0b419778bd958cda69bfff47ba8/cf2, priority=9, startTime=1676015672940; duration=0sec 2023-02-10 07:54:32,993 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:32,999 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af-top 2023-02-10 07:54:33,002 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,002 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24-top 2023-02-10 07:54:33,005 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404-top 2023-02-10 07:54:33,008 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216-top 2023-02-10 07:54:33,011 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:33,012 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:33,014 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:33,014 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:33,018 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:33,018 DEBUG [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:33,018 INFO [StoreOpener-ecec840173cb92d09b34c16fdbcc87bb-1] regionserver.HStore(310): Store=ecec840173cb92d09b34c16fdbcc87bb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:33,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:33,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:33,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:54:33,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:33,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ecec840173cb92d09b34c16fdbcc87bb; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64045005, jitterRate=-0.045655056834220886}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:54:33,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ecec840173cb92d09b34c16fdbcc87bb: Running coprocessor pre-open hook at 1676015672944Writing region info on filesystem at 1676015672944Initializing all the Stores at 1676015672945 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015672945Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015672945Cleaning up temporary data from old regions at 1676015673020 (+75 ms)Cleaning up detritus from prior splits at 1676015673020Running coprocessor post-open hooks at 1676015673023 (+3 ms)Region opened successfully at 1676015673023 2023-02-10 07:54:33,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb., pid=77, masterSystemTime=1676015672857 2023-02-10 07:54:33,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:33,025 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:33,025 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:54:33,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:33,025 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,025 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,025 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1912): ecec840173cb92d09b34c16fdbcc87bb/cf1 is initiating minor compaction (all files) 2023-02-10 07:54:33,025 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1912): ecec840173cb92d09b34c16fdbcc87bb/cf2 is initiating minor compaction (all files) 2023-02-10 07:54:33,026 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ecec840173cb92d09b34c16fdbcc87bb/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,026 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ecec840173cb92d09b34c16fdbcc87bb/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,026 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-95013f8f2be74bb5a707cf27d4c41d8e, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-256d2d76498749fab5f4a1bffd7af7be, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8366be06fb074a05aa9774621b31a305] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp, totalSize=34.8 K 2023-02-10 07:54:33,026 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-c0d9c8c781dd4429837b1042675de285, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-bc6fb538ab704ec4892c06a0d1419890, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-0bb230ae00c3463b8a7fdcb4c304dfd8] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp, totalSize=34.8 K 2023-02-10 07:54:33,026 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting 404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015670543 2023-02-10 07:54:33,026 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015670543 2023-02-10 07:54:33,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,027 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015670734 2023-02-10 07:54:33,027 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015670734 2023-02-10 07:54:33,027 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=ecec840173cb92d09b34c16fdbcc87bb, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:33,027 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015670914 2023-02-10 07:54:33,027 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015673027"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015673027"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015673027"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015673027"}]},"ts":"1676015673027"} 2023-02-10 07:54:33,027 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015670914 2023-02-10 07:54:33,028 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676015671638 2023-02-10 07:54:33,028 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676015671638 2023-02-10 07:54:33,028 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-95013f8f2be74bb5a707cf27d4c41d8e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,029 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-c0d9c8c781dd4429837b1042675de285, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,029 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-256d2d76498749fab5f4a1bffd7af7be, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,029 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-bc6fb538ab704ec4892c06a0d1419890, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,029 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8366be06fb074a05aa9774621b31a305, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,029 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-0bb230ae00c3463b8a7fdcb4c304dfd8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-10 07:54:33,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure ecec840173cb92d09b34c16fdbcc87bb, server=jenkins-hbase4.apache.org,43281,1676015645662 in 323 msec 2023-02-10 07:54:33,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-10 07:54:33,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ecec840173cb92d09b34c16fdbcc87bb, ASSIGN in 481 msec 2023-02-10 07:54:33,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb in 882 msec 2023-02-10 07:54:33,045 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] throttle.PressureAwareThroughputController(145): ecec840173cb92d09b34c16fdbcc87bb#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:33,045 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] throttle.PressureAwareThroughputController(145): ecec840173cb92d09b34c16fdbcc87bb#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:33,067 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 056da62bd38e45dbb3fee9fdce00d80f: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,068 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp/cf2/056da62bd38e45dbb3fee9fdce00d80f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/056da62bd38e45dbb3fee9fdce00d80f 2023-02-10 07:54:33,069 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3db82213a2684367b36a56d1f0237456: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,070 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp/cf1/3db82213a2684367b36a56d1f0237456 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/3db82213a2684367b36a56d1f0237456 2023-02-10 07:54:33,072 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 056da62bd38e45dbb3fee9fdce00d80f: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,073 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ecec840173cb92d09b34c16fdbcc87bb/cf2 of ecec840173cb92d09b34c16fdbcc87bb into 056da62bd38e45dbb3fee9fdce00d80f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:33,073 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ecec840173cb92d09b34c16fdbcc87bb: 2023-02-10 07:54:33,073 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb., storeName=ecec840173cb92d09b34c16fdbcc87bb/cf2, priority=9, startTime=1676015673025; duration=0sec 2023-02-10 07:54:33,073 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:33,073 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 3db82213a2684367b36a56d1f0237456: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,074 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ecec840173cb92d09b34c16fdbcc87bb/cf1 of ecec840173cb92d09b34c16fdbcc87bb into 3db82213a2684367b36a56d1f0237456(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:33,074 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ecec840173cb92d09b34c16fdbcc87bb: 2023-02-10 07:54:33,074 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb., storeName=ecec840173cb92d09b34c16fdbcc87bb/cf1, priority=9, startTime=1676015673024; duration=0sec 2023-02-10 07:54:33,074 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:33,164 DEBUG [Listener at localhost/43795] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-10 07:54:33,167 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37420, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-10 07:54:33,167 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:33,168 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-10 07:54:33,168 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] regionserver.HStore(1912): e55fa0b419778bd958cda69bfff47ba8/cf1 is initiating major compaction (all files) 2023-02-10 07:54:33,168 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7908972e; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:33,168 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e55fa0b419778bd958cda69bfff47ba8/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:33,169 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-10 07:54:33,169 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/4b71e1b1a51c48d4bddac1d8303ad10d] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp, totalSize=6.8 K 2023-02-10 07:54:33,169 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] regionserver.HStore(1912): e55fa0b419778bd958cda69bfff47ba8/cf2 is initiating major compaction (all files) 2023-02-10 07:54:33,169 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@a8f7fbe; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:33,169 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e55fa0b419778bd958cda69bfff47ba8/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:33,169 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/ad5814dc2e3242abb9ee3824b935b49f] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp, totalSize=6.8 K 2023-02-10 07:54:33,169 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting 4b71e1b1a51c48d4bddac1d8303ad10d, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015670543 2023-02-10 07:54:33,169 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting ad5814dc2e3242abb9ee3824b935b49f, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015670543 2023-02-10 07:54:33,170 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,171 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-10 07:54:33,171 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] regionserver.HStore(1912): ecec840173cb92d09b34c16fdbcc87bb/cf1 is initiating major compaction (all files) 2023-02-10 07:54:33,171 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@20177aa6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:33,171 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-10 07:54:33,171 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] regionserver.HStore(1912): ecec840173cb92d09b34c16fdbcc87bb/cf2 is initiating major compaction (all files) 2023-02-10 07:54:33,171 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1c5fec79; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-10 07:54:33,172 INFO [Listener at localhost/43795] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-10 07:54:33,174 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] throttle.PressureAwareThroughputController(145): e55fa0b419778bd958cda69bfff47ba8#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:33,174 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] throttle.PressureAwareThroughputController(145): e55fa0b419778bd958cda69bfff47ba8#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:33,182 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. is not mergeable because it has references 2023-02-10 07:54:33,183 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. is not mergeable because it has references 2023-02-10 07:54:33,197 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp/cf2/edd2038efac64bfd8f30bdcee5865775 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/edd2038efac64bfd8f30bdcee5865775 2023-02-10 07:54:33,198 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/.tmp/cf1/ffe23c623253490aa1f19eb266015f8c as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/ffe23c623253490aa1f19eb266015f8c 2023-02-10 07:54:33,201 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e55fa0b419778bd958cda69bfff47ba8/cf2 of e55fa0b419778bd958cda69bfff47ba8 into edd2038efac64bfd8f30bdcee5865775(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:33,202 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e55fa0b419778bd958cda69bfff47ba8: 2023-02-10 07:54:33,203 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e55fa0b419778bd958cda69bfff47ba8/cf1 of e55fa0b419778bd958cda69bfff47ba8 into ffe23c623253490aa1f19eb266015f8c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:33,203 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e55fa0b419778bd958cda69bfff47ba8: 2023-02-10 07:54:33,203 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676015673168; duration=0sec 2023-02-10 07:54:33,203 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676015673169; duration=0sec 2023-02-10 07:54:33,203 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-10 07:54:33,203 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:54:33,203 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ecec840173cb92d09b34c16fdbcc87bb/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,203 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ecec840173cb92d09b34c16fdbcc87bb/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,203 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/056da62bd38e45dbb3fee9fdce00d80f] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp, totalSize=5.0 K 2023-02-10 07:54:33,203 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/3db82213a2684367b36a56d1f0237456] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp, totalSize=5.0 K 2023-02-10 07:54:33,204 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] compactions.Compactor(207): Compacting 056da62bd38e45dbb3fee9fdce00d80f, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,204 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] compactions.Compactor(207): Compacting 3db82213a2684367b36a56d1f0237456, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-10 07:54:33,206 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] throttle.PressureAwareThroughputController(145): ecec840173cb92d09b34c16fdbcc87bb#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:33,206 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] throttle.PressureAwareThroughputController(145): ecec840173cb92d09b34c16fdbcc87bb#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:54:33,221 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for bb457ae48899430e9908421dc7dc97f2: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,221 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c23bc469a14648a18d4c0d2b4cbeaea6: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,222 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp/cf1/bb457ae48899430e9908421dc7dc97f2 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/bb457ae48899430e9908421dc7dc97f2 2023-02-10 07:54:33,222 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/.tmp/cf2/c23bc469a14648a18d4c0d2b4cbeaea6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/c23bc469a14648a18d4c0d2b4cbeaea6 2023-02-10 07:54:33,225 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for bb457ae48899430e9908421dc7dc97f2: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,226 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c23bc469a14648a18d4c0d2b4cbeaea6: NONE, but ROW specified in column family configuration 2023-02-10 07:54:33,226 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in ecec840173cb92d09b34c16fdbcc87bb/cf1 of ecec840173cb92d09b34c16fdbcc87bb into bb457ae48899430e9908421dc7dc97f2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:33,226 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ecec840173cb92d09b34c16fdbcc87bb: 2023-02-10 07:54:33,226 INFO [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676015673171; duration=0sec 2023-02-10 07:54:33,226 DEBUG [RS:0;jenkins-hbase4:43281-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:33,226 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in ecec840173cb92d09b34c16fdbcc87bb/cf2 of ecec840173cb92d09b34c16fdbcc87bb into c23bc469a14648a18d4c0d2b4cbeaea6(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:54:33,227 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ecec840173cb92d09b34c16fdbcc87bb: 2023-02-10 07:54:33,227 INFO [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676015673171; duration=0sec 2023-02-10 07:54:33,227 DEBUG [RS:0;jenkins-hbase4:43281-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:54:33,245 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:54:33,290 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=43281] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. is not mergeable because it has references 2023-02-10 07:54:33,291 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=43281] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. is not mergeable because it has references 2023-02-10 07:54:33,297 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-10 07:54:33,299 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:33,300 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,301 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-10 07:54:33,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015673305"}]},"ts":"1676015673305"} 2023-02-10 07:54:33,306 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-10 07:54:33,308 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-10 07:54:33,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e55fa0b419778bd958cda69bfff47ba8, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ecec840173cb92d09b34c16fdbcc87bb, UNASSIGN}] 2023-02-10 07:54:33,309 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ecec840173cb92d09b34c16fdbcc87bb, UNASSIGN 2023-02-10 07:54:33,310 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ecec840173cb92d09b34c16fdbcc87bb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:33,310 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015673310"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015673310"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015673310"}]},"ts":"1676015673310"} 2023-02-10 07:54:33,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure ecec840173cb92d09b34c16fdbcc87bb, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:33,313 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e55fa0b419778bd958cda69bfff47ba8, UNASSIGN 2023-02-10 07:54:33,314 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=e55fa0b419778bd958cda69bfff47ba8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:54:33,314 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015673314"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015673314"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015673314"}]},"ts":"1676015673314"} 2023-02-10 07:54:33,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure e55fa0b419778bd958cda69bfff47ba8, server=jenkins-hbase4.apache.org,43281,1676015645662}] 2023-02-10 07:54:33,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-10 07:54:33,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:33,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ecec840173cb92d09b34c16fdbcc87bb, disabling compactions & flushes 2023-02-10 07:54:33,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. after waiting 0 ms 2023-02-10 07:54:33,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,464 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-95013f8f2be74bb5a707cf27d4c41d8e, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-256d2d76498749fab5f4a1bffd7af7be, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/3db82213a2684367b36a56d1f0237456, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8366be06fb074a05aa9774621b31a305] to archive 2023-02-10 07:54:33,464 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:33,466 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,467 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,468 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,469 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,470 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-95013f8f2be74bb5a707cf27d4c41d8e to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:33,471 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-256d2d76498749fab5f4a1bffd7af7be to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:33,472 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/3db82213a2684367b36a56d1f0237456 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/3db82213a2684367b36a56d1f0237456 2023-02-10 07:54:33,473 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8366be06fb074a05aa9774621b31a305 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:33,475 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-c0d9c8c781dd4429837b1042675de285, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-bc6fb538ab704ec4892c06a0d1419890, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/056da62bd38e45dbb3fee9fdce00d80f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-0bb230ae00c3463b8a7fdcb4c304dfd8] to archive 2023-02-10 07:54:33,475 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:33,476 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,477 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,478 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,479 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,480 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-c0d9c8c781dd4429837b1042675de285 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:33,481 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-bc6fb538ab704ec4892c06a0d1419890 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:33,482 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/056da62bd38e45dbb3fee9fdce00d80f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/056da62bd38e45dbb3fee9fdce00d80f 2023-02-10 07:54:33,484 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-0bb230ae00c3463b8a7fdcb4c304dfd8 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:33,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-10 07:54:33,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:33,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb. 2023-02-10 07:54:33,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ecec840173cb92d09b34c16fdbcc87bb: Waiting for close lock at 1676015673463Running coprocessor pre-close hooks at 1676015673463Disabling compacts and flushes for region at 1676015673463Disabling writes for close at 1676015673463Writing region close event to WAL at 1676015673484 (+21 ms)Running coprocessor post-close hooks at 1676015673487 (+3 ms)Closed at 1676015673488 (+1 ms) 2023-02-10 07:54:33,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:33,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:33,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e55fa0b419778bd958cda69bfff47ba8, disabling compactions & flushes 2023-02-10 07:54:33,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:33,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:33,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. after waiting 0 ms 2023-02-10 07:54:33,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:33,490 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ecec840173cb92d09b34c16fdbcc87bb, regionState=CLOSED 2023-02-10 07:54:33,490 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015673490"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015673490"}]},"ts":"1676015673490"} 2023-02-10 07:54:33,491 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-6111193081ef4fdcb3a9c353d4b0ea58, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1c0466ba52da4059bf3208f176c434df, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-744ea1a98ace46eb98c8596a9d6b204f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/4b71e1b1a51c48d4bddac1d8303ad10d, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c-bottom] to archive 2023-02-10 07:54:33,492 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:33,493 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/404ccfb3ca4741e8a71deb9f03198342.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-10 07:54:33,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure ecec840173cb92d09b34c16fdbcc87bb, server=jenkins-hbase4.apache.org,43281,1676015645662 in 181 msec 2023-02-10 07:54:33,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ecec840173cb92d09b34c16fdbcc87bb, UNASSIGN in 185 msec 2023-02-10 07:54:33,494 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/a7971f39a5bf4765aa12f17df2550926.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,496 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/d17d3670df374a748707bce015d1cda8.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,496 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-6111193081ef4fdcb3a9c353d4b0ea58 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-6111193081ef4fdcb3a9c353d4b0ea58 2023-02-10 07:54:33,497 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1c0466ba52da4059bf3208f176c434df to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1c0466ba52da4059bf3208f176c434df 2023-02-10 07:54:33,498 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-744ea1a98ace46eb98c8596a9d6b204f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-744ea1a98ace46eb98c8596a9d6b204f 2023-02-10 07:54:33,499 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/4b71e1b1a51c48d4bddac1d8303ad10d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/4b71e1b1a51c48d4bddac1d8303ad10d 2023-02-10 07:54:33,500 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/e23ea22ec97f47c3afe664026051c49c.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,502 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-35ae71e6675241e2b1cca138df49975b, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8b31ef218cf4459fa3297d31bf9fa8d6, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1744974f9a2a41aba80883aa57b3fdce, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/ad5814dc2e3242abb9ee3824b935b49f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24-bottom] to archive 2023-02-10 07:54:33,502 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:54:33,504 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/8d7f9da32de149ceb43b4f6354a06404.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,504 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/3428adab207e43daa350eb819529c3af.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,505 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/aa6387c3b0e0496d9c762ad9b5f9b216.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,506 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-35ae71e6675241e2b1cca138df49975b to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-35ae71e6675241e2b1cca138df49975b 2023-02-10 07:54:33,507 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8b31ef218cf4459fa3297d31bf9fa8d6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-8b31ef218cf4459fa3297d31bf9fa8d6 2023-02-10 07:54:33,508 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1744974f9a2a41aba80883aa57b3fdce to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/testSplitTableRegionDeletedRowsDaughter=09f620010a784524f1f7bb2817b44a9f-1744974f9a2a41aba80883aa57b3fdce 2023-02-10 07:54:33,508 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/ad5814dc2e3242abb9ee3824b935b49f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/ad5814dc2e3242abb9ee3824b935b49f 2023-02-10 07:54:33,509 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/406b8e1f39f645188641a4fdd8c8bb24.09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-10 07:54:33,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:33,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8. 2023-02-10 07:54:33,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e55fa0b419778bd958cda69bfff47ba8: Waiting for close lock at 1676015673490Running coprocessor pre-close hooks at 1676015673490Disabling compacts and flushes for region at 1676015673490Disabling writes for close at 1676015673490Writing region close event to WAL at 1676015673510 (+20 ms)Running coprocessor post-close hooks at 1676015673512 (+2 ms)Closed at 1676015673513 (+1 ms) 2023-02-10 07:54:33,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:33,515 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=e55fa0b419778bd958cda69bfff47ba8, regionState=CLOSED 2023-02-10 07:54:33,515 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676015673515"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015673515"}]},"ts":"1676015673515"} 2023-02-10 07:54:33,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-10 07:54:33,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure e55fa0b419778bd958cda69bfff47ba8, server=jenkins-hbase4.apache.org,43281,1676015645662 in 201 msec 2023-02-10 07:54:33,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-10 07:54:33,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e55fa0b419778bd958cda69bfff47ba8, UNASSIGN in 209 msec 2023-02-10 07:54:33,519 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015673519"}]},"ts":"1676015673519"} 2023-02-10 07:54:33,520 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-10 07:54:33,521 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-10 07:54:33,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 221 msec 2023-02-10 07:54:33,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-10 07:54:33,606 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-10 07:54:33,607 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,609 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,610 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-10 07:54:33,614 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,614 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:33,614 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:33,616 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/recovered.edits] 2023-02-10 07:54:33,616 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/recovered.edits] 2023-02-10 07:54:33,617 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/recovered.edits] 2023-02-10 07:54:33,620 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/ffe23c623253490aa1f19eb266015f8c to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf1/ffe23c623253490aa1f19eb266015f8c 2023-02-10 07:54:33,620 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/bb457ae48899430e9908421dc7dc97f2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf1/bb457ae48899430e9908421dc7dc97f2 2023-02-10 07:54:33,621 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/1c0466ba52da4059bf3208f176c434df to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/1c0466ba52da4059bf3208f176c434df 2023-02-10 07:54:33,622 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/256d2d76498749fab5f4a1bffd7af7be to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/256d2d76498749fab5f4a1bffd7af7be 2023-02-10 07:54:33,623 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/edd2038efac64bfd8f30bdcee5865775 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/cf2/edd2038efac64bfd8f30bdcee5865775 2023-02-10 07:54:33,623 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/c23bc469a14648a18d4c0d2b4cbeaea6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/cf2/c23bc469a14648a18d4c0d2b4cbeaea6 2023-02-10 07:54:33,623 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/404ccfb3ca4741e8a71deb9f03198342 2023-02-10 07:54:33,625 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/6111193081ef4fdcb3a9c353d4b0ea58 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/6111193081ef4fdcb3a9c353d4b0ea58 2023-02-10 07:54:33,626 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/744ea1a98ace46eb98c8596a9d6b204f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/744ea1a98ace46eb98c8596a9d6b204f 2023-02-10 07:54:33,626 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/recovered.edits/113.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8/recovered.edits/113.seqid 2023-02-10 07:54:33,626 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/recovered.edits/113.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb/recovered.edits/113.seqid 2023-02-10 07:54:33,627 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e55fa0b419778bd958cda69bfff47ba8 2023-02-10 07:54:33,627 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:54:33,627 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/8366be06fb074a05aa9774621b31a305 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/8366be06fb074a05aa9774621b31a305 2023-02-10 07:54:33,628 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/95013f8f2be74bb5a707cf27d4c41d8e to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/95013f8f2be74bb5a707cf27d4c41d8e 2023-02-10 07:54:33,630 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/a7971f39a5bf4765aa12f17df2550926 2023-02-10 07:54:33,631 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/d17d3670df374a748707bce015d1cda8 2023-02-10 07:54:33,632 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf1/e23ea22ec97f47c3afe664026051c49c 2023-02-10 07:54:33,634 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/0bb230ae00c3463b8a7fdcb4c304dfd8 2023-02-10 07:54:33,635 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/1744974f9a2a41aba80883aa57b3fdce to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/1744974f9a2a41aba80883aa57b3fdce 2023-02-10 07:54:33,637 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/3428adab207e43daa350eb819529c3af 2023-02-10 07:54:33,638 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/35ae71e6675241e2b1cca138df49975b to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/35ae71e6675241e2b1cca138df49975b 2023-02-10 07:54:33,639 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/406b8e1f39f645188641a4fdd8c8bb24 2023-02-10 07:54:33,640 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8b31ef218cf4459fa3297d31bf9fa8d6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8b31ef218cf4459fa3297d31bf9fa8d6 2023-02-10 07:54:33,641 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/8d7f9da32de149ceb43b4f6354a06404 2023-02-10 07:54:33,642 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/aa6387c3b0e0496d9c762ad9b5f9b216 2023-02-10 07:54:33,643 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/bc6fb538ab704ec4892c06a0d1419890 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/bc6fb538ab704ec4892c06a0d1419890 2023-02-10 07:54:33,645 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/c0d9c8c781dd4429837b1042675de285 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/cf2/c0d9c8c781dd4429837b1042675de285 2023-02-10 07:54:33,647 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/recovered.edits/106.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f/recovered.edits/106.seqid 2023-02-10 07:54:33,648 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/09f620010a784524f1f7bb2817b44a9f 2023-02-10 07:54:33,648 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-10 07:54:33,650 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,652 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-10 07:54:33,654 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-10 07:54:33,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-10 07:54:33,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015673655"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:33,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015673655"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:33,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015673655"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:33,657 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-10 07:54:33,657 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 09f620010a784524f1f7bb2817b44a9f, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676015670039.09f620010a784524f1f7bb2817b44a9f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e55fa0b419778bd958cda69bfff47ba8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676015672152.e55fa0b419778bd958cda69bfff47ba8.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => ecec840173cb92d09b34c16fdbcc87bb, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676015672152.ecec840173cb92d09b34c16fdbcc87bb.', STARTKEY => '60', ENDKEY => ''}] 2023-02-10 07:54:33,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-10 07:54:33,657 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015673657"}]},"ts":"9223372036854775807"} 2023-02-10 07:54:33,658 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-10 07:54:33,661 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:54:33,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 54 msec 2023-02-10 07:54:33,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-10 07:54:33,711 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-10 07:54:33,725 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=411 (was 406) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:59232 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:59866 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:51746 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:55712 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:58940 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:51668 [Waiting for operation #42] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:58780 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:58688 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:55718 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:60320 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:58714 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:52532 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:58438 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:58924 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:58698 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_708221192_15 at /127.0.0.1:33182 [Waiting for operation #44] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=888 (was 886) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=246 (was 233) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3414 (was 3483) 2023-02-10 07:54:33,741 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=411, OpenFileDescriptor=888, MaxFileDescriptor=60000, SystemLoadAverage=246, ProcessCount=168, AvailableMemoryMB=3413 2023-02-10 07:54:33,741 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:54:33,744 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:54:33,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:54:33,745 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:54:33,747 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:54:33,748 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:54:33,749 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:33,750 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d empty. 2023-02-10 07:54:33,750 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:33,750 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-10 07:54:33,765 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:33,766 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8e2fb35af8b7ceda6eef617b8936238d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:54:33,774 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:33,774 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 8e2fb35af8b7ceda6eef617b8936238d, disabling compactions & flushes 2023-02-10 07:54:33,774 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:33,774 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:33,774 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. after waiting 0 ms 2023-02-10 07:54:33,774 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:33,774 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:33,774 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 8e2fb35af8b7ceda6eef617b8936238d: Waiting for close lock at 1676015673774Disabling compacts and flushes for region at 1676015673774Disabling writes for close at 1676015673774Writing region close event to WAL at 1676015673774Closed at 1676015673774 2023-02-10 07:54:33,776 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:54:33,777 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015673777"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015673777"}]},"ts":"1676015673777"} 2023-02-10 07:54:33,778 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:54:33,779 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:54:33,779 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015673779"}]},"ts":"1676015673779"} 2023-02-10 07:54:33,780 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-10 07:54:33,782 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:33,783 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:33,783 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:33,783 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:33,783 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:54:33,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, ASSIGN}] 2023-02-10 07:54:33,784 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, ASSIGN 2023-02-10 07:54:33,785 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:54:33,935 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:33,937 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:33,938 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015673937"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015673937"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015673937"}]},"ts":"1676015673937"} 2023-02-10 07:54:33,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:34,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8e2fb35af8b7ceda6eef617b8936238d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:34,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:34,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:34,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,098 INFO [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,099 DEBUG [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/cf1 2023-02-10 07:54:34,099 DEBUG [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/cf1 2023-02-10 07:54:34,100 INFO [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8e2fb35af8b7ceda6eef617b8936238d columnFamilyName cf1 2023-02-10 07:54:34,100 INFO [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] regionserver.HStore(310): Store=8e2fb35af8b7ceda6eef617b8936238d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:34,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,104 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:54:34,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8e2fb35af8b7ceda6eef617b8936238d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71006641, jitterRate=0.058081403374671936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 07:54:34,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8e2fb35af8b7ceda6eef617b8936238d: Running coprocessor pre-open hook at 1676015674097Writing region info on filesystem at 1676015674097Initializing all the Stores at 1676015674098 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015674098Cleaning up temporary data from old regions at 1676015674102 (+4 ms)Cleaning up detritus from prior splits at 1676015674102Running coprocessor post-open hooks at 1676015674106 (+4 ms)Region opened successfully at 1676015674106 2023-02-10 07:54:34,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d., pid=87, masterSystemTime=1676015674090 2023-02-10 07:54:34,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,108 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:34,108 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015674108"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015674108"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015674108"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015674108"}]},"ts":"1676015674108"} 2023-02-10 07:54:34,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-10 07:54:34,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736 in 170 msec 2023-02-10 07:54:34,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-10 07:54:34,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, ASSIGN in 327 msec 2023-02-10 07:54:34,112 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:54:34,112 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015674112"}]},"ts":"1676015674112"} 2023-02-10 07:54:34,113 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-10 07:54:34,119 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:54:34,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 374 msec 2023-02-10 07:54:34,254 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-10 07:54:34,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:54:34,272 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-10 07:54:34,273 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-10 07:54:34,274 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-10 07:54:34,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-10 07:54:34,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, REOPEN/MOVE}] 2023-02-10 07:54:34,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, REOPEN/MOVE 2023-02-10 07:54:34,297 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:34,297 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015674297"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015674297"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015674297"}]},"ts":"1676015674297"} 2023-02-10 07:54:34,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:34,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8e2fb35af8b7ceda6eef617b8936238d, disabling compactions & flushes 2023-02-10 07:54:34,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. after waiting 0 ms 2023-02-10 07:54:34,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-10 07:54:34,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:54:34,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8e2fb35af8b7ceda6eef617b8936238d: Waiting for close lock at 1676015674450Running coprocessor pre-close hooks at 1676015674450Disabling compacts and flushes for region at 1676015674450Disabling writes for close at 1676015674451 (+1 ms)Writing region close event to WAL at 1676015674452 (+1 ms)Running coprocessor post-close hooks at 1676015674455 (+3 ms)Closed at 1676015674455 2023-02-10 07:54:34,455 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 8e2fb35af8b7ceda6eef617b8936238d to self. 2023-02-10 07:54:34,457 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,457 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=CLOSED 2023-02-10 07:54:34,457 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015674457"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015674457"}]},"ts":"1676015674457"} 2023-02-10 07:54:34,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-10 07:54:34,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736 in 159 msec 2023-02-10 07:54:34,460 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=true 2023-02-10 07:54:34,610 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:54:34,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:34,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015674611"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015674611"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015674611"}]},"ts":"1676015674611"} 2023-02-10 07:54:34,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:54:34,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8e2fb35af8b7ceda6eef617b8936238d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:54:34,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:34,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:34,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,768 INFO [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,769 DEBUG [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/cf1 2023-02-10 07:54:34,769 DEBUG [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/cf1 2023-02-10 07:54:34,770 INFO [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8e2fb35af8b7ceda6eef617b8936238d columnFamilyName cf1 2023-02-10 07:54:34,770 INFO [StoreOpener-8e2fb35af8b7ceda6eef617b8936238d-1] regionserver.HStore(310): Store=8e2fb35af8b7ceda6eef617b8936238d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:54:34,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:54:34,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8e2fb35af8b7ceda6eef617b8936238d; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63558410, jitterRate=-0.052905887365341187}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 07:54:34,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8e2fb35af8b7ceda6eef617b8936238d: Running coprocessor pre-open hook at 1676015674767Writing region info on filesystem at 1676015674767Initializing all the Stores at 1676015674768 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015674768Cleaning up temporary data from old regions at 1676015674772 (+4 ms)Cleaning up detritus from prior splits at 1676015674772Running coprocessor post-open hooks at 1676015674774 (+2 ms)Region opened successfully at 1676015674774 2023-02-10 07:54:34,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d., pid=92, masterSystemTime=1676015674764 2023-02-10 07:54:34,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:54:34,776 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:54:34,776 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015674776"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015674776"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015674776"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015674776"}]},"ts":"1676015674776"} 2023-02-10 07:54:34,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-10 07:54:34,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736 in 166 msec 2023-02-10 07:54:34,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-10 07:54:34,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, REOPEN/MOVE in 483 msec 2023-02-10 07:54:34,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-10 07:54:34,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 499 msec 2023-02-10 07:54:34,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc, ASSIGN}] 2023-02-10 07:54:34,787 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc, ASSIGN 2023-02-10 07:54:34,787 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-10 07:54:34,938 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:54:34,939 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:54:34,939 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:54:34,939 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:54:34,939 DEBUG [jenkins-hbase4:37109] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-10 07:54:34,940 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=738761cd8c8642035da172e02506c7fc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:54:34,940 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015674940"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676015674940"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676015674940"}]},"ts":"1676015674940"} 2023-02-10 07:54:34,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 738761cd8c8642035da172e02506c7fc, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:54:35,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. 2023-02-10 07:54:35,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 738761cd8c8642035da172e02506c7fc, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-10 07:54:35,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:54:35,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 738761cd8c8642035da172e02506c7fc 2023-02-10 07:54:35,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:54:35,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 738761cd8c8642035da172e02506c7fc 2023-02-10 07:54:35,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 738761cd8c8642035da172e02506c7fc 2023-02-10 07:54:35,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:35,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:36,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,063 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-10 07:54:37,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:37,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,363 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:54:38,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:38,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:39,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:40,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:41,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:42,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:43,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:44,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:45,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:46,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:47,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:48,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:49,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:50,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:51,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:52,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:53,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:54,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:55,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:56,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:57,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:58,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:54:59,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:00,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:01,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:02,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:03,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:04,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:05,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:06,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,048 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-10 07:55:07,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,065 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-10 07:55:07,120 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-10 07:55:07,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:07,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:08,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:09,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:10,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:11,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:12,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:13,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:14,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:15,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:16,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:17,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:18,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:19,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:20,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:21,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:22,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:23,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:24,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:25,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:26,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:27,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:28,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:29,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:30,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:31,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:32,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:33,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-10 07:55:34,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-10 07:55:34,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-10 07:55:34,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:34,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-10 07:55:35,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-10 07:55:35,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:35,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,471 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:36,472 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:36,473 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:36,473 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:36,473 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:36,475 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 8e2fb35af8b7ceda6eef617b8936238d, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:36,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 exec-time=6 msec 2023-02-10 07:55:36,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-10 07:55:36,819 INFO [StoreOpener-738761cd8c8642035da172e02506c7fc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 738761cd8c8642035da172e02506c7fc 2023-02-10 07:55:36,823 DEBUG [StoreOpener-738761cd8c8642035da172e02506c7fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/cf1 2023-02-10 07:55:36,823 DEBUG [StoreOpener-738761cd8c8642035da172e02506c7fc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/cf1 2023-02-10 07:55:36,823 INFO [StoreOpener-738761cd8c8642035da172e02506c7fc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 738761cd8c8642035da172e02506c7fc columnFamilyName cf1 2023-02-10 07:55:36,824 INFO [StoreOpener-738761cd8c8642035da172e02506c7fc-1] regionserver.HStore(310): Store=738761cd8c8642035da172e02506c7fc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:36,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 738761cd8c8642035da172e02506c7fc; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72298150, jitterRate=0.07732638716697693}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-10 07:55:36,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 738761cd8c8642035da172e02506c7fc: Running coprocessor pre-open hook at 1676015675099Initializing all the Stores at 1676015736819 (+61720 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015736819Running coprocessor post-open hooks at 1676015736826 (+7 ms)Region opened successfully at 1676015736826 2023-02-10 07:55:36,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc., pid=94, masterSystemTime=1676015675095 2023-02-10 07:55:36,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. 2023-02-10 07:55:36,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. 2023-02-10 07:55:36,827 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=738761cd8c8642035da172e02506c7fc, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:36,827 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015736827"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676015736827"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676015736827"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676015736827"}]},"ts":"1676015736827"} 2023-02-10 07:55:36,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-10 07:55:36,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 738761cd8c8642035da172e02506c7fc, server=jenkins-hbase4.apache.org,43211,1676015645706 in 1 mins, 1.887 sec 2023-02-10 07:55:36,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-10 07:55:36,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc, ASSIGN in 1 mins, 2.044 sec 2023-02-10 07:55:36,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.576 sec 2023-02-10 07:55:36,834 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676015674251","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62582,"client":"172.31.14.131:44470","queuetimems":0,"class":"HMaster"} 2023-02-10 07:55:36,834 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-10 07:55:38,723 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up 2023-02-10 07:55:38,724 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:38,726 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:38,726 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:38,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:38,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-10 07:55:38,730 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015738729"}]},"ts":"1676015738729"} 2023-02-10 07:55:38,731 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-10 07:55:38,732 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-10 07:55:38,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc, UNASSIGN}] 2023-02-10 07:55:38,733 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc, UNASSIGN 2023-02-10 07:55:38,734 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=738761cd8c8642035da172e02506c7fc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:38,734 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015738734"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676015738734"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676015738734"}]},"ts":"1676015738734"} 2023-02-10 07:55:38,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 738761cd8c8642035da172e02506c7fc, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:55:38,737 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, UNASSIGN 2023-02-10 07:55:38,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:38,738 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015738737"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015738737"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015738737"}]},"ts":"1676015738737"} 2023-02-10 07:55:38,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:38,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-10 07:55:38,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 738761cd8c8642035da172e02506c7fc 2023-02-10 07:55:38,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 738761cd8c8642035da172e02506c7fc, disabling compactions & flushes 2023-02-10 07:55:38,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. 2023-02-10 07:55:38,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. 2023-02-10 07:55:38,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. after waiting 0 ms 2023-02-10 07:55:38,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. 2023-02-10 07:55:38,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:38,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc. 2023-02-10 07:55:38,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 738761cd8c8642035da172e02506c7fc: Waiting for close lock at 1676015738887Running coprocessor pre-close hooks at 1676015738887Disabling compacts and flushes for region at 1676015738887Disabling writes for close at 1676015738888 (+1 ms)Writing region close event to WAL at 1676015738888Running coprocessor post-close hooks at 1676015738889 (+1 ms)Closed at 1676015738889 2023-02-10 07:55:38,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:55:38,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8e2fb35af8b7ceda6eef617b8936238d, disabling compactions & flushes 2023-02-10 07:55:38,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:55:38,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:55:38,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. after waiting 0 ms 2023-02-10 07:55:38,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:55:38,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 738761cd8c8642035da172e02506c7fc 2023-02-10 07:55:38,894 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=738761cd8c8642035da172e02506c7fc, regionState=CLOSED 2023-02-10 07:55:38,894 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015738892"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676015738892"}]},"ts":"1676015738892"} 2023-02-10 07:55:38,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-10 07:55:38,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:38,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d. 2023-02-10 07:55:38,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-10 07:55:38,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8e2fb35af8b7ceda6eef617b8936238d: Waiting for close lock at 1676015738890Running coprocessor pre-close hooks at 1676015738890Disabling compacts and flushes for region at 1676015738890Disabling writes for close at 1676015738891 (+1 ms)Writing region close event to WAL at 1676015738891Running coprocessor post-close hooks at 1676015738899 (+8 ms)Closed at 1676015738899 2023-02-10 07:55:38,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 738761cd8c8642035da172e02506c7fc, server=jenkins-hbase4.apache.org,43211,1676015645706 in 162 msec 2023-02-10 07:55:38,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:55:38,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc, UNASSIGN in 166 msec 2023-02-10 07:55:38,901 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=8e2fb35af8b7ceda6eef617b8936238d, regionState=CLOSED 2023-02-10 07:55:38,901 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676015738901"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015738901"}]},"ts":"1676015738901"} 2023-02-10 07:55:38,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-10 07:55:38,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 8e2fb35af8b7ceda6eef617b8936238d, server=jenkins-hbase4.apache.org,40867,1676015645736 in 164 msec 2023-02-10 07:55:38,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-10 07:55:38,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=8e2fb35af8b7ceda6eef617b8936238d, UNASSIGN in 170 msec 2023-02-10 07:55:38,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015738905"}]},"ts":"1676015738905"} 2023-02-10 07:55:38,906 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-10 07:55:38,908 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-10 07:55:38,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 182 msec 2023-02-10 07:55:39,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-10 07:55:39,031 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-10 07:55:39,032 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:39,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:39,034 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:39,035 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:39,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-10 07:55:39,040 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:55:39,041 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/recovered.edits] 2023-02-10 07:55:39,046 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/recovered.edits/7.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d/recovered.edits/7.seqid 2023-02-10 07:55:39,046 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/8e2fb35af8b7ceda6eef617b8936238d 2023-02-10 07:55:39,046 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-10 07:55:39,048 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:39,051 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-10 07:55:39,053 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-10 07:55:39,053 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:39,054 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-10 07:55:39,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015739054"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:39,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015739054"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:39,055 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-10 07:55:39,055 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8e2fb35af8b7ceda6eef617b8936238d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676015673745.8e2fb35af8b7ceda6eef617b8936238d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 738761cd8c8642035da172e02506c7fc, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676015673745_0001.738761cd8c8642035da172e02506c7fc.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-10 07:55:39,055 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-10 07:55:39,055 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015739055"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:39,057 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-10 07:55:39,058 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:39,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 26 msec 2023-02-10 07:55:39,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-10 07:55:39,136 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-10 07:55:39,148 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=347 (was 411), OpenFileDescriptor=653 (was 888), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=95 (was 246), ProcessCount=168 (was 168), AvailableMemoryMB=3215 (was 3413) 2023-02-10 07:55:39,160 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=348, OpenFileDescriptor=653, MaxFileDescriptor=60000, SystemLoadAverage=95, ProcessCount=168, AvailableMemoryMB=3214 2023-02-10 07:55:39,160 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:39,163 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:55:39,164 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:55:39,165 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:39,166 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:55:39,167 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:55:39,168 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,169 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0 empty. 2023-02-10 07:55:39,169 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,169 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-10 07:55:39,178 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-10 07:55:39,179 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 51a0b6341417d9df603913d3c5f8a2b0, NAME => 'testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:55:39,186 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:39,186 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 51a0b6341417d9df603913d3c5f8a2b0, disabling compactions & flushes 2023-02-10 07:55:39,186 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,186 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,186 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. after waiting 0 ms 2023-02-10 07:55:39,186 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,186 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,186 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 51a0b6341417d9df603913d3c5f8a2b0: Waiting for close lock at 1676015739186Disabling compacts and flushes for region at 1676015739186Disabling writes for close at 1676015739186Writing region close event to WAL at 1676015739186Closed at 1676015739186 2023-02-10 07:55:39,188 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:55:39,189 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676015739189"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015739189"}]},"ts":"1676015739189"} 2023-02-10 07:55:39,190 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:55:39,190 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:55:39,191 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015739191"}]},"ts":"1676015739191"} 2023-02-10 07:55:39,192 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-10 07:55:39,194 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:55:39,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:55:39,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:55:39,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:55:39,195 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:55:39,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=51a0b6341417d9df603913d3c5f8a2b0, ASSIGN}] 2023-02-10 07:55:39,196 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=51a0b6341417d9df603913d3c5f8a2b0, ASSIGN 2023-02-10 07:55:39,197 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=51a0b6341417d9df603913d3c5f8a2b0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:55:39,347 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:55:39,348 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=51a0b6341417d9df603913d3c5f8a2b0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,348 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676015739348"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015739348"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015739348"}]},"ts":"1676015739348"} 2023-02-10 07:55:39,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 51a0b6341417d9df603913d3c5f8a2b0, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:39,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 51a0b6341417d9df603913d3c5f8a2b0, NAME => 'testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:55:39,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:39,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:39,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,507 INFO [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,509 DEBUG [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1 2023-02-10 07:55:39,509 DEBUG [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1 2023-02-10 07:55:39,509 INFO [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51a0b6341417d9df603913d3c5f8a2b0 columnFamilyName cf1 2023-02-10 07:55:39,510 INFO [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] regionserver.HStore(310): Store=51a0b6341417d9df603913d3c5f8a2b0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:39,510 INFO [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,511 DEBUG [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2 2023-02-10 07:55:39,511 DEBUG [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2 2023-02-10 07:55:39,512 INFO [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51a0b6341417d9df603913d3c5f8a2b0 columnFamilyName cf2 2023-02-10 07:55:39,512 INFO [StoreOpener-51a0b6341417d9df603913d3c5f8a2b0-1] regionserver.HStore(310): Store=51a0b6341417d9df603913d3c5f8a2b0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:39,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:39,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:39,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:55:39,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 51a0b6341417d9df603913d3c5f8a2b0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66303596, jitterRate=-0.011999428272247314}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:39,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 51a0b6341417d9df603913d3c5f8a2b0: Running coprocessor pre-open hook at 1676015739506Writing region info on filesystem at 1676015739506Initializing all the Stores at 1676015739506Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015739506Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015739506Cleaning up temporary data from old regions at 1676015739514 (+8 ms)Cleaning up detritus from prior splits at 1676015739514Running coprocessor post-open hooks at 1676015739518 (+4 ms)Region opened successfully at 1676015739518 2023-02-10 07:55:39,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0., pid=104, masterSystemTime=1676015739501 2023-02-10 07:55:39,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,521 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=51a0b6341417d9df603913d3c5f8a2b0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,521 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676015739521"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015739521"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015739521"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015739521"}]},"ts":"1676015739521"} 2023-02-10 07:55:39,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-10 07:55:39,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 51a0b6341417d9df603913d3c5f8a2b0, server=jenkins-hbase4.apache.org,40867,1676015645736 in 172 msec 2023-02-10 07:55:39,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-10 07:55:39,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=51a0b6341417d9df603913d3c5f8a2b0, ASSIGN in 329 msec 2023-02-10 07:55:39,526 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:55:39,526 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015739526"}]},"ts":"1676015739526"} 2023-02-10 07:55:39,527 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-10 07:55:39,529 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:55:39,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 365 msec 2023-02-10 07:55:39,672 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:39,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-10 07:55:39,674 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-10 07:55:39,674 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,674 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-10 07:55:39,674 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-10 07:55:39,674 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,674 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,677 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,677 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,677 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,677 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,677 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,677 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:39,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,679 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:39,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:39,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,679 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:39,679 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:39,679 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,679 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. started... 2023-02-10 07:55:39,680 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 51a0b6341417d9df603913d3c5f8a2b0 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:55:39,681 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,682 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,682 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,682 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:39,682 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,682 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:39,690 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/cf204c9196284b159594fec9fc144337 2023-02-10 07:55:39,711 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/5bd8220df36b419f97095cbffddefbe9 2023-02-10 07:55:39,715 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/cf204c9196284b159594fec9fc144337 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/cf204c9196284b159594fec9fc144337 2023-02-10 07:55:39,719 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/cf204c9196284b159594fec9fc144337, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:39,720 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/5bd8220df36b419f97095cbffddefbe9 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5bd8220df36b419f97095cbffddefbe9 2023-02-10 07:55:39,724 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5bd8220df36b419f97095cbffddefbe9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:39,725 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 51a0b6341417d9df603913d3c5f8a2b0 in 45ms, sequenceid=6, compaction requested=false 2023-02-10 07:55:39,725 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-10 07:55:39,726 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 51a0b6341417d9df603913d3c5f8a2b0: 2023-02-10 07:55:39,726 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,726 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:39,726 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:39,726 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,726 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:39,726 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:39,728 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,728 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:39,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:39,728 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,728 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:39,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:39,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:39,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:39,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-10 07:55:39,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44f8e20e[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:39,730 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-10 07:55:39,731 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:39,733 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,733 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:39,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:39,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:39,733 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:39,735 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:39,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:39,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,735 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:39,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:39,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:39,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:39,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-10 07:55:39,739 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-10 07:55:39,739 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:39,739 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:39,740 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-10 07:55:39,740 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:39,741 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,741 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:39,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,741 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:39,742 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,742 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,743 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:39,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,745 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:39,746 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,746 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,750 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,750 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,750 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,750 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,751 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:39,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,751 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:39,751 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,751 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,751 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,751 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,851 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:39,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:39,863 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:39,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:39,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-10 07:55:39,865 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-10 07:55:39,865 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,865 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-10 07:55:39,865 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-10 07:55:39,865 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,865 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,866 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,866 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,866 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,866 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,867 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,867 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:39,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:39,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:39,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:39,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:39,868 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:39,868 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,868 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:39,868 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:39,868 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:39,869 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:39,869 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:39,869 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,869 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:39,870 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:39,870 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,870 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. started... 2023-02-10 07:55:39,871 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 51a0b6341417d9df603913d3c5f8a2b0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:39,875 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,876 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,876 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:39,881 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/c480f8f6db914ba0b88c94fdb96600c5 2023-02-10 07:55:39,895 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/3a024e29a1da4ef3b721df5108e03a92 2023-02-10 07:55:39,899 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/c480f8f6db914ba0b88c94fdb96600c5 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/c480f8f6db914ba0b88c94fdb96600c5 2023-02-10 07:55:39,903 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/c480f8f6db914ba0b88c94fdb96600c5, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:39,904 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/3a024e29a1da4ef3b721df5108e03a92 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/3a024e29a1da4ef3b721df5108e03a92 2023-02-10 07:55:39,908 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/3a024e29a1da4ef3b721df5108e03a92, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:39,909 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 51a0b6341417d9df603913d3c5f8a2b0 in 39ms, sequenceid=19, compaction requested=false 2023-02-10 07:55:39,909 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 51a0b6341417d9df603913d3c5f8a2b0: 2023-02-10 07:55:39,909 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:39,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:39,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:39,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:39,909 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:39,911 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,911 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:39,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:39,911 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,911 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:39,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:39,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-10 07:55:39,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66699d1[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:39,914 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-10 07:55:39,914 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:39,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,915 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:39,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,915 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:39,915 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:39,916 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:39,915 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:39,916 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:39,915 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:39,916 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:39,916 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:39,917 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:39,917 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,917 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:39,917 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:39,917 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,917 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:39,920 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,920 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:39,920 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:39,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:39,920 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:39,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:39,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:39,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-10 07:55:39,924 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-10 07:55:39,924 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:39,924 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:39,924 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-10 07:55:39,924 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:39,926 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,926 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:39,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:39,926 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,927 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:39,928 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:39,929 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:39,930 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,930 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,933 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:39,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:39,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:39,933 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-10 07:55:39,934 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:39,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:39,933 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:39,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,934 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:39,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:39,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:39,934 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:40,034 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:40,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:40,046 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:40,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:40,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-10 07:55:40,048 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-10 07:55:40,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-10 07:55:40,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-10 07:55:40,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,049 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,049 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,049 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,049 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,050 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,050 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,051 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,052 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:40,052 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:40,052 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,052 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. started... 2023-02-10 07:55:40,052 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 51a0b6341417d9df603913d3c5f8a2b0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:40,054 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,054 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,054 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,054 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,055 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,055 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,062 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/a97d98f26a09488bafad265040f8791d 2023-02-10 07:55:40,073 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/5d19225334a84385879a71d914b6291f 2023-02-10 07:55:40,076 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/a97d98f26a09488bafad265040f8791d as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/a97d98f26a09488bafad265040f8791d 2023-02-10 07:55:40,079 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/a97d98f26a09488bafad265040f8791d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:40,080 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/5d19225334a84385879a71d914b6291f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5d19225334a84385879a71d914b6291f 2023-02-10 07:55:40,083 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5d19225334a84385879a71d914b6291f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:40,084 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 51a0b6341417d9df603913d3c5f8a2b0 in 32ms, sequenceid=32, compaction requested=true 2023-02-10 07:55:40,084 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 51a0b6341417d9df603913d3c5f8a2b0: 2023-02-10 07:55:40,084 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,084 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:40,084 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:40,084 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,084 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,084 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,086 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,086 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,086 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,086 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-10 07:55:40,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@df7ffdc[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:40,088 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-10 07:55:40,088 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,089 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,089 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,089 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,090 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:40,090 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:40,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:40,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,090 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:40,092 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,092 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,092 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,092 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,093 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,093 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,093 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,093 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,093 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,093 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:40,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-10 07:55:40,096 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-10 07:55:40,096 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:40,096 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:40,096 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-10 07:55:40,096 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:40,097 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,098 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,098 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,098 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,099 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,100 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,100 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,101 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,106 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-10 07:55:40,107 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:40,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,106 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:40,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,107 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,108 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,108 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,207 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:40,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:40,220 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:40,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:40,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-10 07:55:40,222 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-10 07:55:40,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-10 07:55:40,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-10 07:55:40,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,223 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,223 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,223 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,224 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,225 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,226 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,226 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:40,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:40,226 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,226 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. started... 2023-02-10 07:55:40,226 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 51a0b6341417d9df603913d3c5f8a2b0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:40,227 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,228 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,238 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/423a4808f27e4df1a17c671e08baf2ce 2023-02-10 07:55:40,249 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/a67f846734c347728de87b56b7dbee9f 2023-02-10 07:55:40,253 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/423a4808f27e4df1a17c671e08baf2ce as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/423a4808f27e4df1a17c671e08baf2ce 2023-02-10 07:55:40,257 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/423a4808f27e4df1a17c671e08baf2ce, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:40,257 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/a67f846734c347728de87b56b7dbee9f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/a67f846734c347728de87b56b7dbee9f 2023-02-10 07:55:40,261 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/a67f846734c347728de87b56b7dbee9f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:40,262 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 51a0b6341417d9df603913d3c5f8a2b0 in 36ms, sequenceid=45, compaction requested=true 2023-02-10 07:55:40,262 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 51a0b6341417d9df603913d3c5f8a2b0: 2023-02-10 07:55:40,262 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:40,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:40,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,265 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,265 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,266 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,266 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-10 07:55:40,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35ed5303[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:40,268 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-10 07:55:40,268 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,269 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:40,269 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,269 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,269 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:40,270 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,270 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,270 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:40,270 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,270 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,270 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:40,271 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,271 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,271 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,271 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,271 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,271 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,271 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,273 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,273 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,273 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:40,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-10 07:55:40,276 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-10 07:55:40,276 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:40,277 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:40,277 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-10 07:55:40,277 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:40,278 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,278 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,278 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,279 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,279 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,279 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,280 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,282 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,282 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,282 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,285 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-10 07:55:40,285 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,285 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,285 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,286 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,286 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:40,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,286 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,286 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,286 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,286 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,287 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,287 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,286 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:40,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,387 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:40,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:40,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:40,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:40,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-10 07:55:40,400 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-10 07:55:40,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-10 07:55:40,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-10 07:55:40,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,401 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,401 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,401 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,401 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,401 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,402 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,402 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:40,403 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:40,403 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,403 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. started... 2023-02-10 07:55:40,404 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 51a0b6341417d9df603913d3c5f8a2b0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:40,408 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,409 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,409 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,409 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,409 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,409 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,412 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/37615226a2e646158f7b84d8d9ce194d 2023-02-10 07:55:40,421 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/19437c151bbf4ff587cad80e313dfe24 2023-02-10 07:55:40,425 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/37615226a2e646158f7b84d8d9ce194d as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/37615226a2e646158f7b84d8d9ce194d 2023-02-10 07:55:40,428 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/37615226a2e646158f7b84d8d9ce194d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:40,429 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/19437c151bbf4ff587cad80e313dfe24 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/19437c151bbf4ff587cad80e313dfe24 2023-02-10 07:55:40,433 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/19437c151bbf4ff587cad80e313dfe24, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:40,433 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 51a0b6341417d9df603913d3c5f8a2b0 in 29ms, sequenceid=58, compaction requested=true 2023-02-10 07:55:40,433 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 51a0b6341417d9df603913d3c5f8a2b0: 2023-02-10 07:55:40,433 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,433 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:40,434 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:40,434 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,434 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,434 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,435 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,435 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,435 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,435 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-10 07:55:40,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36863f22[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:40,437 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-10 07:55:40,437 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,438 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:40,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,439 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:40,438 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:40,444 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,444 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,444 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,444 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,444 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,444 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,444 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,447 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,447 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,447 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:40,450 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-10 07:55:40,450 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-10 07:55:40,450 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:40,450 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:40,450 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-10 07:55:40,450 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:40,451 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,451 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,451 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,452 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,451 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,452 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,452 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,454 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,454 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,454 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,457 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,457 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-10 07:55:40,457 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,457 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,458 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,458 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,458 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,458 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,458 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,557 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:40,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:40,569 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:40,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:40,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-10 07:55:40,570 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-10 07:55:40,570 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,570 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-10 07:55:40,570 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-10 07:55:40,571 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,571 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,574 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,574 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,574 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,574 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,574 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,574 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:40,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,576 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:40,576 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:40,576 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,576 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. started... 2023-02-10 07:55:40,577 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 51a0b6341417d9df603913d3c5f8a2b0 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:40,577 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,578 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,578 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,578 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,578 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,578 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,584 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/81382fdb53714fd1a406814659e7030f 2023-02-10 07:55:40,594 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/35b8efc34e3543d2bf76de8fbf7548ac 2023-02-10 07:55:40,597 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/81382fdb53714fd1a406814659e7030f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/81382fdb53714fd1a406814659e7030f 2023-02-10 07:55:40,600 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/81382fdb53714fd1a406814659e7030f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:40,601 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/35b8efc34e3543d2bf76de8fbf7548ac as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/35b8efc34e3543d2bf76de8fbf7548ac 2023-02-10 07:55:40,604 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/35b8efc34e3543d2bf76de8fbf7548ac, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:40,605 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 51a0b6341417d9df603913d3c5f8a2b0 in 28ms, sequenceid=71, compaction requested=true 2023-02-10 07:55:40,605 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 51a0b6341417d9df603913d3c5f8a2b0: 2023-02-10 07:55:40,605 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,605 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:40,605 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:40,605 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,606 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-10 07:55:40,606 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-10 07:55:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,607 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-10 07:55:40,609 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49c8032[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:40,609 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-10 07:55:40,609 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:40,611 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,611 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:40,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:40,611 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:40,612 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,612 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,612 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,613 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,613 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,613 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-10 07:55:40,613 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,613 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,613 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,613 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-10 07:55:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:40,616 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-10 07:55:40,616 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-10 07:55:40,616 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:40,616 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:40,616 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-10 07:55:40,616 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:40,617 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,617 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:40,617 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,618 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:40,618 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,618 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,619 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:40,620 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-10 07:55:40,620 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,621 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:40,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-10 07:55:40,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,624 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:40,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-10 07:55:40,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:40,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:40,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:40,624 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:40,625 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:40,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,625 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:40,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:40,625 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,625 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:40,625 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-10 07:55:40,625 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:40,625 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-10 07:55:40,725 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:40,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:40,733 INFO [Listener at localhost/43795] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,734 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.StoreUtils(129): cannot split hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/c480f8f6db914ba0b88c94fdb96600c5 because midkey is the same as first or last row 2023-02-10 07:55:40,734 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.StoreUtils(129): cannot split hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/3a024e29a1da4ef3b721df5108e03a92 because midkey is the same as first or last row 2023-02-10 07:55:40,734 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.HRegion(2744): Flushing 51a0b6341417d9df603913d3c5f8a2b0 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-10 07:55:40,744 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/5dd423c9a39f4ec3a10962e8a00a2477 2023-02-10 07:55:40,753 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/f6b4e618bc8441eb81b05e40a00d55de 2023-02-10 07:55:40,757 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf1/5dd423c9a39f4ec3a10962e8a00a2477 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/5dd423c9a39f4ec3a10962e8a00a2477 2023-02-10 07:55:40,760 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/5dd423c9a39f4ec3a10962e8a00a2477, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:40,761 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/.tmp/cf2/f6b4e618bc8441eb81b05e40a00d55de as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/f6b4e618bc8441eb81b05e40a00d55de 2023-02-10 07:55:40,764 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/f6b4e618bc8441eb81b05e40a00d55de, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:40,765 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 51a0b6341417d9df603913d3c5f8a2b0 in 31ms, sequenceid=82, compaction requested=true 2023-02-10 07:55:40,766 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.HRegion(2445): Flush status journal for 51a0b6341417d9df603913d3c5f8a2b0: 2023-02-10 07:55:40,766 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.StoreUtils(129): cannot split hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/c480f8f6db914ba0b88c94fdb96600c5 because midkey is the same as first or last row 2023-02-10 07:55:40,766 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40867] regionserver.StoreUtils(129): cannot split hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/3a024e29a1da4ef3b721df5108e03a92 because midkey is the same as first or last row 2023-02-10 07:55:40,766 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,766 DEBUG [Listener at localhost/43795] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-10 07:55:40,766 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:40,768 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-10 07:55:40,768 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-10 07:55:40,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:40,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-10 07:55:40,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015740771"}]},"ts":"1676015740771"} 2023-02-10 07:55:40,772 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-10 07:55:40,773 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-10 07:55:40,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=51a0b6341417d9df603913d3c5f8a2b0, UNASSIGN}] 2023-02-10 07:55:40,774 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=51a0b6341417d9df603913d3c5f8a2b0, UNASSIGN 2023-02-10 07:55:40,775 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=51a0b6341417d9df603913d3c5f8a2b0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:40,775 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676015740775"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015740775"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015740775"}]},"ts":"1676015740775"} 2023-02-10 07:55:40,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 51a0b6341417d9df603913d3c5f8a2b0, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:40,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-10 07:55:40,927 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:40,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 51a0b6341417d9df603913d3c5f8a2b0, disabling compactions & flushes 2023-02-10 07:55:40,928 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. after waiting 0 ms 2023-02-10 07:55:40,928 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-10 07:55:40,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:40,932 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0. 2023-02-10 07:55:40,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 51a0b6341417d9df603913d3c5f8a2b0: Waiting for close lock at 1676015740928Running coprocessor pre-close hooks at 1676015740928Disabling compacts and flushes for region at 1676015740928Disabling writes for close at 1676015740928Writing region close event to WAL at 1676015740928Running coprocessor post-close hooks at 1676015740932 (+4 ms)Closed at 1676015740932 2023-02-10 07:55:40,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:40,934 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=51a0b6341417d9df603913d3c5f8a2b0, regionState=CLOSED 2023-02-10 07:55:40,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676015740934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015740934"}]},"ts":"1676015740934"} 2023-02-10 07:55:40,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-10 07:55:40,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 51a0b6341417d9df603913d3c5f8a2b0, server=jenkins-hbase4.apache.org,40867,1676015645736 in 159 msec 2023-02-10 07:55:40,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-10 07:55:40,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=51a0b6341417d9df603913d3c5f8a2b0, UNASSIGN in 162 msec 2023-02-10 07:55:40,938 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015740938"}]},"ts":"1676015740938"} 2023-02-10 07:55:40,939 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-10 07:55:40,940 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-10 07:55:40,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 173 msec 2023-02-10 07:55:41,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-10 07:55:41,073 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-10 07:55:41,074 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-10 07:55:41,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:41,076 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:41,076 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:41,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-10 07:55:41,079 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:41,081 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/recovered.edits] 2023-02-10 07:55:41,084 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/37615226a2e646158f7b84d8d9ce194d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/37615226a2e646158f7b84d8d9ce194d 2023-02-10 07:55:41,085 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/423a4808f27e4df1a17c671e08baf2ce to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/423a4808f27e4df1a17c671e08baf2ce 2023-02-10 07:55:41,086 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/5dd423c9a39f4ec3a10962e8a00a2477 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/5dd423c9a39f4ec3a10962e8a00a2477 2023-02-10 07:55:41,087 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/81382fdb53714fd1a406814659e7030f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/81382fdb53714fd1a406814659e7030f 2023-02-10 07:55:41,088 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/a97d98f26a09488bafad265040f8791d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/a97d98f26a09488bafad265040f8791d 2023-02-10 07:55:41,089 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/c480f8f6db914ba0b88c94fdb96600c5 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/c480f8f6db914ba0b88c94fdb96600c5 2023-02-10 07:55:41,090 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/cf204c9196284b159594fec9fc144337 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf1/cf204c9196284b159594fec9fc144337 2023-02-10 07:55:41,092 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/19437c151bbf4ff587cad80e313dfe24 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/19437c151bbf4ff587cad80e313dfe24 2023-02-10 07:55:41,093 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/35b8efc34e3543d2bf76de8fbf7548ac to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/35b8efc34e3543d2bf76de8fbf7548ac 2023-02-10 07:55:41,094 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/3a024e29a1da4ef3b721df5108e03a92 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/3a024e29a1da4ef3b721df5108e03a92 2023-02-10 07:55:41,095 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5bd8220df36b419f97095cbffddefbe9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5bd8220df36b419f97095cbffddefbe9 2023-02-10 07:55:41,096 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5d19225334a84385879a71d914b6291f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/5d19225334a84385879a71d914b6291f 2023-02-10 07:55:41,097 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/a67f846734c347728de87b56b7dbee9f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/a67f846734c347728de87b56b7dbee9f 2023-02-10 07:55:41,098 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/f6b4e618bc8441eb81b05e40a00d55de to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/cf2/f6b4e618bc8441eb81b05e40a00d55de 2023-02-10 07:55:41,100 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/recovered.edits/85.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0/recovered.edits/85.seqid 2023-02-10 07:55:41,100 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testInvalidSplitKey/51a0b6341417d9df603913d3c5f8a2b0 2023-02-10 07:55:41,100 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-10 07:55:41,102 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:41,103 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-10 07:55:41,107 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-10 07:55:41,108 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:41,108 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-10 07:55:41,108 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015741108"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:41,109 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-10 07:55:41,109 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 51a0b6341417d9df603913d3c5f8a2b0, NAME => 'testInvalidSplitKey,,1676015739164.51a0b6341417d9df603913d3c5f8a2b0.', STARTKEY => '', ENDKEY => ''}] 2023-02-10 07:55:41,109 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-10 07:55:41,109 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015741109"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:41,110 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-10 07:55:41,113 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:41,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 39 msec 2023-02-10 07:55:41,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-10 07:55:41,178 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-10 07:55:41,191 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=367 (was 348) - Thread LEAK? -, OpenFileDescriptor=740 (was 653) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=95 (was 95), ProcessCount=168 (was 168), AvailableMemoryMB=3209 (was 3214) 2023-02-10 07:55:41,204 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=367, OpenFileDescriptor=740, MaxFileDescriptor=60000, SystemLoadAverage=95, ProcessCount=168, AvailableMemoryMB=3208 2023-02-10 07:55:41,204 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:41,207 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:55:41,207 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:55:41,208 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:41,209 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:55:41,210 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:55:41,211 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,211 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2 empty. 2023-02-10 07:55:41,212 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,212 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-10 07:55:41,220 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-10 07:55:41,221 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7ebee4f2e2077f5c617dbf6b2850d8b2, NAME => 'testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:55:41,228 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:41,229 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 7ebee4f2e2077f5c617dbf6b2850d8b2, disabling compactions & flushes 2023-02-10 07:55:41,229 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,229 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,229 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. after waiting 0 ms 2023-02-10 07:55:41,229 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,229 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,229 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: Waiting for close lock at 1676015741229Disabling compacts and flushes for region at 1676015741229Disabling writes for close at 1676015741229Writing region close event to WAL at 1676015741229Closed at 1676015741229 2023-02-10 07:55:41,230 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:55:41,231 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015741231"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015741231"}]},"ts":"1676015741231"} 2023-02-10 07:55:41,232 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:55:41,233 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:55:41,233 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015741233"}]},"ts":"1676015741233"} 2023-02-10 07:55:41,234 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-10 07:55:41,236 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:55:41,237 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:55:41,237 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:55:41,237 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:55:41,237 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:55:41,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, ASSIGN}] 2023-02-10 07:55:41,238 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, ASSIGN 2023-02-10 07:55:41,239 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43211,1676015645706; forceNewPlan=false, retain=false 2023-02-10 07:55:41,389 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:55:41,390 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,391 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015741390"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015741390"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015741390"}]},"ts":"1676015741390"} 2023-02-10 07:55:41,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:55:41,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7ebee4f2e2077f5c617dbf6b2850d8b2, NAME => 'testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:55:41,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:41,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:41,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,547 INFO [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,549 DEBUG [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1 2023-02-10 07:55:41,549 DEBUG [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1 2023-02-10 07:55:41,549 INFO [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ebee4f2e2077f5c617dbf6b2850d8b2 columnFamilyName cf1 2023-02-10 07:55:41,550 INFO [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] regionserver.HStore(310): Store=7ebee4f2e2077f5c617dbf6b2850d8b2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:41,550 INFO [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,551 DEBUG [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2 2023-02-10 07:55:41,551 DEBUG [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2 2023-02-10 07:55:41,551 INFO [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7ebee4f2e2077f5c617dbf6b2850d8b2 columnFamilyName cf2 2023-02-10 07:55:41,551 INFO [StoreOpener-7ebee4f2e2077f5c617dbf6b2850d8b2-1] regionserver.HStore(310): Store=7ebee4f2e2077f5c617dbf6b2850d8b2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:41,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:41,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:41,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:55:41,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7ebee4f2e2077f5c617dbf6b2850d8b2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73700160, jitterRate=0.09821796417236328}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:41,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: Running coprocessor pre-open hook at 1676015741547Writing region info on filesystem at 1676015741547Initializing all the Stores at 1676015741547Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015741547Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015741547Cleaning up temporary data from old regions at 1676015741553 (+6 ms)Cleaning up detritus from prior splits at 1676015741553Running coprocessor post-open hooks at 1676015741557 (+4 ms)Region opened successfully at 1676015741557 2023-02-10 07:55:41,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2., pid=111, masterSystemTime=1676015741543 2023-02-10 07:55:41,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,559 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,559 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015741559"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015741559"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015741559"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015741559"}]},"ts":"1676015741559"} 2023-02-10 07:55:41,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-10 07:55:41,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 in 168 msec 2023-02-10 07:55:41,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-10 07:55:41,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, ASSIGN in 325 msec 2023-02-10 07:55:41,564 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:55:41,564 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015741564"}]},"ts":"1676015741564"} 2023-02-10 07:55:41,565 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-10 07:55:41,568 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:55:41,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 361 msec 2023-02-10 07:55:41,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:41,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-10 07:55:41,716 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:41,716 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,716 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-10 07:55:41,716 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:55:41,717 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,717 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,718 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,718 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,718 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,718 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,718 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,719 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:41,719 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:41,720 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,720 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:41,720 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:41,720 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. started... 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:41,720 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:41,721 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:55:41,723 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,723 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,723 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,723 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:41,723 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,723 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:41,732 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/797ff7b0ceab45c483921d3caddb26a5 2023-02-10 07:55:41,744 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/b60e10b0dad546258c63c6de371154a6 2023-02-10 07:55:41,747 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/797ff7b0ceab45c483921d3caddb26a5 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5 2023-02-10 07:55:41,750 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:41,750 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/b60e10b0dad546258c63c6de371154a6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6 2023-02-10 07:55:41,753 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:41,754 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 33ms, sequenceid=6, compaction requested=false 2023-02-10 07:55:41,754 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:41,755 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: 2023-02-10 07:55:41,755 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:41,755 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:41,756 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,756 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:41,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:41,757 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,757 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-10 07:55:41,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36278662[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:41,758 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:55:41,758 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,760 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,760 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,760 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,761 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:41,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,761 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:41,760 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:41,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:41,762 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:41,763 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:41,763 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:41,763 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,763 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:41,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:41,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43211,1676015645706': 2023-02-10 07:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:55:41,765 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-10 07:55:41,766 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:41,766 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:41,766 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,766 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:41,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:41,768 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,768 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:41,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,769 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:41,769 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:41,771 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,771 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,771 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,773 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:41,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:41,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,773 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:41,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,774 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,874 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:41,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:41,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:41,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:41,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-10 07:55:41,886 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:41,886 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,886 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-10 07:55:41,886 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:55:41,886 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,887 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,888 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,888 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,888 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,888 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,888 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,888 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:41,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:41,889 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:41,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:41,890 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,890 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. started... 2023-02-10 07:55:41,890 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:41,891 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,893 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,893 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,893 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:41,893 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,893 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:41,898 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/d68310def3744112b9630c3f0ab1a157 2023-02-10 07:55:41,908 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be 2023-02-10 07:55:41,912 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/d68310def3744112b9630c3f0ab1a157 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157 2023-02-10 07:55:41,915 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:41,916 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be 2023-02-10 07:55:41,919 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:41,919 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 29ms, sequenceid=19, compaction requested=false 2023-02-10 07:55:41,919 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: 2023-02-10 07:55:41,919 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:41,919 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:41,919 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:41,919 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,920 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:41,920 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:41,924 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,924 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:41,924 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,924 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:41,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-10 07:55:41,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@698dbeee[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:41,926 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:55:41,926 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:41,928 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,928 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:41,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,930 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:41,931 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:41,931 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,931 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:41,931 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:41,931 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,931 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:41,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43211,1676015645706': 2023-02-10 07:55:41,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:55:41,934 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-10 07:55:41,934 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:41,934 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:41,934 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-10 07:55:41,934 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:41,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:41,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,936 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:41,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,936 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:41,937 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:41,938 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:41,939 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,939 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:41,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:41,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:41,941 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:41,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:41,942 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:41,942 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:41,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,941 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:41,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:41,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:41,942 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,942 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:41,942 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:41,942 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:41,942 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,042 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:42,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:42,053 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:42,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:42,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-10 07:55:42,054 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,054 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,054 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-10 07:55:42,054 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:55:42,055 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,055 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,057 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,057 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,057 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,057 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,057 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,058 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,058 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,058 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,059 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:42,059 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:42,059 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,059 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. started... 2023-02-10 07:55:42,059 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:42,060 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,060 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,061 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,061 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,061 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,061 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,067 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/4922a75d9b234f70934ff635476918be 2023-02-10 07:55:42,077 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/84741a528c85491ca9ab65911b91bb6c 2023-02-10 07:55:42,080 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/4922a75d9b234f70934ff635476918be as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be 2023-02-10 07:55:42,083 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:42,084 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/84741a528c85491ca9ab65911b91bb6c as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c 2023-02-10 07:55:42,087 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:42,088 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 29ms, sequenceid=32, compaction requested=true 2023-02-10 07:55:42,088 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: 2023-02-10 07:55:42,088 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,088 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,089 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,089 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-10 07:55:42,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ef5f3d2[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:42,091 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:55:42,091 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,093 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,093 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,093 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,093 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,094 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:42,093 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:42,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,094 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,095 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,095 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43211,1676015645706': 2023-02-10 07:55:42,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:55:42,098 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-10 07:55:42,098 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:42,098 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:42,098 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,098 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:42,100 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,100 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,100 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,100 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,101 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,101 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,102 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,102 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,103 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,109 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:42,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,110 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:42,110 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:42,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,210 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:42,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:42,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:42,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:42,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-10 07:55:42,222 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-10 07:55:42,222 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:55:42,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,224 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,224 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,224 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,224 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,224 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,224 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,226 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,232 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. started... 2023-02-10 07:55:42,232 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:42,243 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/bc5ca2d78f694c79b2acd92484a87219 2023-02-10 07:55:42,272 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/8abfcaea20f3486ca0e9ffca5ed2378a 2023-02-10 07:55:42,276 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/bc5ca2d78f694c79b2acd92484a87219 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219 2023-02-10 07:55:42,280 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:42,280 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/8abfcaea20f3486ca0e9ffca5ed2378a as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a 2023-02-10 07:55:42,283 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:42,284 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 52ms, sequenceid=45, compaction requested=true 2023-02-10 07:55:42,284 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: 2023-02-10 07:55:42,284 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,284 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:42,284 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:42,285 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,285 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,285 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,286 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,286 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,287 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,287 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-10 07:55:42,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e08632e[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:42,289 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:55:42,289 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,291 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,291 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,291 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,292 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,292 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:42,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,292 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,293 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,294 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,294 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,294 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,294 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43211,1676015645706': 2023-02-10 07:55:42,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:55:42,296 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-10 07:55:42,296 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:42,297 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:42,297 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,297 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:42,298 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,298 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,298 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,298 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,299 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,299 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,300 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,301 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,301 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,304 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,304 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,304 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,304 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,304 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,304 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,305 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,304 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:42,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,306 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,305 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:42,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,306 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,306 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,406 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:42,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:42,417 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:42,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:42,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-10 07:55:42,418 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,419 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,419 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-10 07:55:42,419 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:55:42,419 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,419 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,420 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,420 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,420 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,420 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,420 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,420 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:42,422 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,422 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. started... 2023-02-10 07:55:42,422 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,423 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,423 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,423 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,423 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:42,424 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,424 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,424 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,424 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,425 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,425 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,431 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/870fddc159ba4cdea847fe4fb45b6091 2023-02-10 07:55:42,441 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 2023-02-10 07:55:42,444 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/870fddc159ba4cdea847fe4fb45b6091 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091 2023-02-10 07:55:42,448 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:42,448 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 2023-02-10 07:55:42,451 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:42,452 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 29ms, sequenceid=58, compaction requested=true 2023-02-10 07:55:42,452 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: 2023-02-10 07:55:42,452 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,452 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:42,452 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:42,452 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,452 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,452 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,456 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,456 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,457 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,457 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-10 07:55:42,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@416f01b[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:42,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:55:42,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:42,460 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,460 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,460 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:42,462 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,462 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,462 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,462 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,462 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,462 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,462 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,462 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43211,1676015645706': 2023-02-10 07:55:42,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:55:42,465 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-10 07:55:42,465 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:42,465 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:42,465 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,465 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:42,467 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,467 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,467 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,467 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,467 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,468 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,469 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,470 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,470 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:42,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,472 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,473 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,473 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,473 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,472 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,473 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,473 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:42,473 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:42,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,473 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,573 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:42,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:42,584 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:42,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:42,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-10 07:55:42,586 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,586 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,586 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-10 07:55:42,586 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-10 07:55:42,586 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,586 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,588 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,588 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,588 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,588 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,589 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,589 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,590 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,590 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,590 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,590 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,590 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-10 07:55:42,591 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:42,591 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:42,591 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,591 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. started... 2023-02-10 07:55:42,591 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:42,591 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,592 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,592 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,592 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,592 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,592 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,598 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/2763456ca0234e6fa0fcb64fd97a2588 2023-02-10 07:55:42,614 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/9cde6baa229c4d918d51e46c54c56e63 2023-02-10 07:55:42,618 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/2763456ca0234e6fa0fcb64fd97a2588 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588 2023-02-10 07:55:42,621 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:42,622 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/9cde6baa229c4d918d51e46c54c56e63 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63 2023-02-10 07:55:42,625 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:42,626 INFO [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 35ms, sequenceid=71, compaction requested=true 2023-02-10 07:55:42,626 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: 2023-02-10 07:55:42,626 DEBUG [rs(jenkins-hbase4.apache.org,43211,1676015645706)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:42,626 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:42,626 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:42,626 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,626 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-10 07:55:42,626 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-10 07:55:42,627 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,627 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,628 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,628 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-10 07:55:42,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e854b9[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:42,630 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-10 07:55:42,630 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,632 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:42,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:42,632 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:42,633 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,633 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,633 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-10 07:55:42,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,634 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-10 07:55:42,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,43211,1676015645706': 2023-02-10 07:55:42,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-10 07:55:42,637 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-10 07:55:42,637 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:42,637 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:42,637 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-10 07:55:42,637 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:42,638 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,638 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,638 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:42,639 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,639 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,640 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:42,640 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:42,641 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-10 07:55:42,642 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,642 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:42,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-10 07:55:42,648 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,648 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:42,649 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:42,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:42,649 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:42,648 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:42,649 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:42,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,649 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:42,649 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:42,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:42,649 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-10 07:55:42,749 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:42,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:43,010 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-10 07:55:43,011 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:43,012 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,013 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,013 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,013 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,014 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:43,014 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:43,014 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:43,262 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,262 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:43,262 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:43,262 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:43,263 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-10 07:55:43,263 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:43,263 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:43,263 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:43,263 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:43,264 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:43,266 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-10 07:55:43,266 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:43,267 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:43,267 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:43,267 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:43,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:43,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,268 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,269 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,269 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:43,269 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,269 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:43,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:43,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,270 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,271 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,271 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:43,271 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,271 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,272 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,272 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:43,272 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,272 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,272 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,273 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,273 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:43,273 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,273 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,273 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,274 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,274 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,274 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:43,274 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,274 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:43,274 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,274 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,274 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,274 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-10 07:55:43,275 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:43,276 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:43,276 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015773276 2023-02-10 07:55:43,278 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,278 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:43,282 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:43,282 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:43,282 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:43,282 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015803282 2023-02-10 07:55:43,282 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015863282 2023-02-10 07:55:43,282 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:55:43,282 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:43,284 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:43,284 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:43,285 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:43,533 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:43,533 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:43,534 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:43,535 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-10 07:55:43,536 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:43,536 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:43,536 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:43,536 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:43,537 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,537 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:43,537 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,537 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,538 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,538 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:43,538 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,538 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,539 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,539 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:43,539 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:43,539 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,540 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,540 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,540 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:43,540 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,541 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,541 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,541 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:43,541 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,542 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,542 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,542 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,542 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:43,542 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,542 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,543 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,543 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,543 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,543 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:43,543 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,543 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:43,543 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,544 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,544 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,544 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-10 07:55:43,544 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:43,545 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:43,545 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015773545 2023-02-10 07:55:43,547 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,547 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:43,550 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:43,550 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:43,550 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:43,550 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015803550 2023-02-10 07:55:43,550 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015863550 2023-02-10 07:55:43,550 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:55:43,550 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:43,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN}] 2023-02-10 07:55:43,552 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:43,552 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:43,553 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:43,800 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,800 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:43,801 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:43,801 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:43,801 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:43,801 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:43,801 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:43,801 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:43,801 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:43,802 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:43,803 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-10 07:55:43,803 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:43,803 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:43,804 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:43,804 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:43,804 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,804 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:43,804 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,804 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:43,805 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:43,806 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,807 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:43,807 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,807 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,807 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:43,807 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,808 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:43,808 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,808 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,808 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:43,809 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:43,809 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,809 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:43,809 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,809 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,809 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,810 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-10 07:55:43,810 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:43,811 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:43,811 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015773811 2023-02-10 07:55:43,812 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:43,812 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:43,815 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:43,815 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:43,815 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:43,815 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015803815 2023-02-10 07:55:43,815 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015863815 2023-02-10 07:55:43,815 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:55:43,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN}] 2023-02-10 07:55:43,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:43,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:43,818 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:43,818 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015743818"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015743818"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015743818"}]},"ts":"1676015743818"} 2023-02-10 07:55:43,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:55:43,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:43,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7ebee4f2e2077f5c617dbf6b2850d8b2, disabling compactions & flushes 2023-02-10 07:55:43,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:43,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:43,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. after waiting 0 ms 2023-02-10 07:55:43,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:43,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-10 07:55:43,979 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/bedaaf6912494fa68f9787fc7dd5afd4 2023-02-10 07:55:43,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/6b34b649159d4f06bf67b6523534c2aa 2023-02-10 07:55:43,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf1/bedaaf6912494fa68f9787fc7dd5afd4 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4 2023-02-10 07:55:43,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:43,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/.tmp/cf2/6b34b649159d4f06bf67b6523534c2aa as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa 2023-02-10 07:55:44,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:44,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 31ms, sequenceid=82, compaction requested=true 2023-02-10 07:55:44,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-10 07:55:44,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:44,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. 2023-02-10 07:55:44,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7ebee4f2e2077f5c617dbf6b2850d8b2: Waiting for close lock at 1676015743970Running coprocessor pre-close hooks at 1676015743970Disabling compacts and flushes for region at 1676015743970Disabling writes for close at 1676015743971 (+1 ms)Obtaining lock to block concurrent updates at 1676015743971Preparing flush snapshotting stores in 7ebee4f2e2077f5c617dbf6b2850d8b2 at 1676015743971Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676015743971Flushing stores of testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. at 1676015743971Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf1: creating writer at 1676015743972 (+1 ms)Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf1: appending metadata at 1676015743974 (+2 ms)Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf1: closing flushed file at 1676015743974Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf2: creating writer at 1676015743983 (+9 ms)Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf2: appending metadata at 1676015743984 (+1 ms)Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf2: closing flushed file at 1676015743984Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf1: reopening flushed file at 1676015743994 (+10 ms)Flushing 7ebee4f2e2077f5c617dbf6b2850d8b2/cf2: reopening flushed file at 1676015743998 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7ebee4f2e2077f5c617dbf6b2850d8b2 in 31ms, sequenceid=82, compaction requested=true at 1676015744002 (+4 ms)Writing region close event to WAL at 1676015744003 (+1 ms)Running coprocessor post-close hooks at 1676015744006 (+3 ms)Closed at 1676015744008 (+2 ms) 2023-02-10 07:55:44,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,010 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED 2023-02-10 07:55:44,010 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015744010"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015744010"}]},"ts":"1676015744010"} 2023-02-10 07:55:44,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-10 07:55:44,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 in 192 msec 2023-02-10 07:55:44,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-10 07:55:44,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN in 198 msec 2023-02-10 07:55:44,016 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:44,016 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:44,016 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:44,065 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,065 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:44,065 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:44,065 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:44,066 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:44,066 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:44,066 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:44,066 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:44,066 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:44,066 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:44,068 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-10 07:55:44,068 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:44,068 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:44,068 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:44,068 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:44,068 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:44,068 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:44,069 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,069 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:44,069 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,069 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,069 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,069 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:44,070 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,070 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,070 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,070 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:44,070 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:44,070 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,070 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,071 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,071 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:44,071 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,071 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,071 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,071 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:44,071 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,072 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,072 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,072 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,072 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:44,072 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,072 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:44,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:44,073 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:44,074 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:44,074 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,074 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,074 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,074 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:44,074 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:44,074 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-10 07:55:44,074 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:44,075 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:44,075 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015774075 2023-02-10 07:55:44,075 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:44,077 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,079 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:44,079 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:44,079 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:44,079 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015804079 2023-02-10 07:55:44,079 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015864079 2023-02-10 07:55:44,079 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:55:44,082 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:44,090 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=7ebee4f2e2077f5c617dbf6b2850d8b2, threads=14 2023-02-10 07:55:44,090 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,090 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,090 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,091 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,091 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,091 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,091 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,091 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,091 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,091 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,092 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,092 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,092 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,092 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,228 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,236 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,237 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,239 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,248 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,248 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,249 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,267 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,268 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,271 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,272 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,272 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,272 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,274 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,274 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 7ebee4f2e2077f5c617dbf6b2850d8b2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:55:44,311 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:44,311 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:44,311 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:44,330 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:44,330 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:44,331 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:44,332 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-10 07:55:44,332 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:44,332 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:44,332 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:44,333 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:44,333 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:44,333 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:44,333 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,333 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:44,333 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,333 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:44,334 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,335 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,335 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,335 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:44,335 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,335 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,335 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,336 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:44,336 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,336 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,336 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,337 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,337 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:44,337 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,337 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,338 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,338 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,338 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,338 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:44,338 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,339 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:44,339 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:44,339 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:44,339 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,339 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,339 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,339 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:44,339 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:44,340 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-10 07:55:44,340 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:44,341 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:44,341 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015774341 2023-02-10 07:55:44,341 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:44,343 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,345 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:44,345 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:44,345 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:44,345 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015804345 2023-02-10 07:55:44,345 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015864345 2023-02-10 07:55:44,345 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:55:44,353 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=7ebee4f2e2077f5c617dbf6b2850d8b2, threads=14 2023-02-10 07:55:44,354 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,355 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,355 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,356 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,358 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,358 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,358 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,358 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,359 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,359 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,360 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,364 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,364 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,364 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,430 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:55:44,434 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,533 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,534 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,552 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,556 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,608 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,612 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,614 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,615 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,616 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,617 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,626 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,628 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,632 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4 for region: 7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:44,633 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 7ebee4f2e2077f5c617dbf6b2850d8b2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:55:44,664 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:44,668 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:55:44,669 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:55:44,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:44,670 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:44,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:44,846 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:44,846 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:44,847 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:44,848 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-10 07:55:44,848 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:44,848 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:44,848 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:44,849 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:44,849 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:44,849 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:44,849 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,849 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:44,850 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,850 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:44,850 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,850 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:44,850 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,851 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:44,851 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,851 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:44,851 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:44,851 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,852 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:44,852 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,852 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:44,852 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,852 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:44,853 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,853 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:44,853 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,853 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:44,853 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,854 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,854 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:44,854 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,854 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:44,854 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,854 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,855 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:44,855 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:44,855 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,855 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:44,855 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:44,855 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:44,856 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,856 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,856 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,856 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:44,856 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:44,856 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-10 07:55:44,857 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:44,857 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:44,858 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015774858 2023-02-10 07:55:44,859 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,859 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:44,861 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:44,861 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:44,862 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:44,862 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015804862 2023-02-10 07:55:44,862 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015864862 2023-02-10 07:55:44,862 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-10 07:55:44,864 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:44,866 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:44,866 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:44,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:44,866 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:45,112 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,112 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:45,112 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:45,112 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:45,113 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:45,113 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:45,113 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:45,113 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:45,113 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:45,113 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:45,114 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-10 07:55:45,115 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:45,115 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:45,115 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:45,115 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:45,115 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:45,115 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:45,115 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,116 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,117 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,118 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:45,119 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:45,120 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,120 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,120 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,120 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,120 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,120 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:55:45,120 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:45,121 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:45,121 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015775121 2023-02-10 07:55:45,121 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:45,123 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,125 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:45,125 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:45,125 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:45,125 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015805125 2023-02-10 07:55:45,125 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015865125 2023-02-10 07:55:45,125 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:55:45,125 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:45,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015745127"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676015745127"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676015745127"}]},"ts":"1676015745127"} 2023-02-10 07:55:45,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015745127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015745127"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015745127"}]},"ts":"1676015745127"} 2023-02-10 07:55:45,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015745127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015745127"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015745127"}]},"ts":"1676015745127"} 2023-02-10 07:55:45,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:45,129 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:45,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:45,354 ERROR [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1326607998-172.31.14.131-1676015641262 (Datanode Uuid 929ae40a-10a1-4838-a0c1-b6853964e24b) service to localhost/127.0.0.1:34635 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-10 07:55:45,375 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,375 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:45,376 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:45,376 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:45,376 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:45,376 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:45,376 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:45,376 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:45,376 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:45,377 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:45,378 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-10 07:55:45,378 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:45,378 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:45,378 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:45,378 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:45,378 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:45,379 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:45,379 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,379 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:45,379 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,379 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,379 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,379 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:45,380 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,380 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,380 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,380 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:45,380 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,381 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:45,382 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,382 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,382 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,382 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,382 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:45,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:45,383 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,384 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:45,384 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:45,384 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:45,384 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,384 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,384 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,384 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,384 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,384 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-10 07:55:45,385 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:45,385 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:45,385 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015775385 2023-02-10 07:55:45,385 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:45,387 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,389 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:45,389 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:45,389 INFO [Listener at localhost/43795] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:45,390 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=46141c32a6ecc87193e20300b9ff0918, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-10 07:55:45,390 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=bb5e738408977bf961a4dfb6fec29b21, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-10 07:55:45,390 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:45,390 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015805390 2023-02-10 07:55:45,390 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015865390 2023-02-10 07:55:45,390 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:55:45,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676015745390"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676015745390"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676015745390"}]},"ts":"1676015745390"} 2023-02-10 07:55:45,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015745390"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015745390"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015745390"}]},"ts":"1676015745390"} 2023-02-10 07:55:45,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015745390"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015745390"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015745390"}]},"ts":"1676015745390"} 2023-02-10 07:55:45,392 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:45,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:45,394 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:45,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:45,640 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,640 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:45,640 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:45,640 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:45,641 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:45,641 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:45,641 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:45,641 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:45,641 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:45,642 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:45,643 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-10 07:55:45,643 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:45,643 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:45,643 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:45,643 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:45,643 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:45,643 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:45,644 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,644 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:45,644 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,644 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,644 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,644 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:45,644 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,645 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,645 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,645 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:45,645 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:45,645 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,645 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,645 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,646 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,647 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:45,648 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,648 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:45,648 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:45,648 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:45,648 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,648 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,648 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,648 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,648 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,648 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:55:45,649 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:45,649 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:45,650 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015775650 2023-02-10 07:55:45,650 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:45,651 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,654 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:45,654 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:45,654 INFO [Listener at localhost/43795] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:45,654 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=46141c32a6ecc87193e20300b9ff0918, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-10 07:55:45,655 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=bb5e738408977bf961a4dfb6fec29b21, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-10 07:55:45,655 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:45,655 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015805655 2023-02-10 07:55:45,655 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015865655 2023-02-10 07:55:45,655 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-10 07:55:45,655 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:45,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, ASSIGN}] 2023-02-10 07:55:45,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:45,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:45,657 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:45,905 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,905 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:45,905 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:45,905 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:45,905 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:45,905 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:45,906 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:45,906 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:45,906 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:45,906 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:45,907 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-10 07:55:45,907 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:45,907 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:45,908 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:45,908 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:45,908 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:45,908 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:45,908 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:45,909 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:45,910 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,911 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:45,912 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:45,913 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,913 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,913 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,913 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,913 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:45,913 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-10 07:55:45,913 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:45,914 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:45,914 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015775914 2023-02-10 07:55:45,915 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:45,915 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:45,918 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:45,918 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:45,918 INFO [Listener at localhost/43795] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:45,918 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=46141c32a6ecc87193e20300b9ff0918, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-10 07:55:45,918 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=bb5e738408977bf961a4dfb6fec29b21, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-10 07:55:45,918 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:45,918 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015805918 2023-02-10 07:55:45,918 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015865918 2023-02-10 07:55:45,918 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:55:45,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, ASSIGN}] 2023-02-10 07:55:45,919 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:45,920 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, ASSIGN 2023-02-10 07:55:45,921 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43211,1676015645706; forceNewPlan=false, retain=false 2023-02-10 07:55:45,921 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, ASSIGN 2023-02-10 07:55:45,922 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,43211,1676015645706; forceNewPlan=false, retain=false 2023-02-10 07:55:46,071 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-10 07:55:46,073 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=bb5e738408977bf961a4dfb6fec29b21, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,073 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015746073"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015746073"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015746073"}]},"ts":"1676015746073"} 2023-02-10 07:55:46,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure bb5e738408977bf961a4dfb6fec29b21, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:55:46,076 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=46141c32a6ecc87193e20300b9ff0918, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,076 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015746076"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015746076"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015746076"}]},"ts":"1676015746076"} 2023-02-10 07:55:46,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 46141c32a6ecc87193e20300b9ff0918, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:55:46,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:46,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 46141c32a6ecc87193e20300b9ff0918, NAME => 'testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.', STARTKEY => '', ENDKEY => '41'} 2023-02-10 07:55:46,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:46,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:46,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,229 INFO [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,230 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1 2023-02-10 07:55:46,230 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1 2023-02-10 07:55:46,231 INFO [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46141c32a6ecc87193e20300b9ff0918 columnFamilyName cf1 2023-02-10 07:55:46,242 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588-bottom 2023-02-10 07:55:46,245 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be-bottom 2023-02-10 07:55:46,248 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5-bottom 2023-02-10 07:55:46,252 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091-bottom 2023-02-10 07:55:46,255 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219-bottom 2023-02-10 07:55:46,258 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4-bottom 2023-02-10 07:55:46,262 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157-bottom 2023-02-10 07:55:46,262 INFO [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(310): Store=46141c32a6ecc87193e20300b9ff0918/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:46,262 INFO [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,263 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2 2023-02-10 07:55:46,263 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2 2023-02-10 07:55:46,263 INFO [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 46141c32a6ecc87193e20300b9ff0918 columnFamilyName cf2 2023-02-10 07:55:46,274 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa-bottom 2023-02-10 07:55:46,278 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c-bottom 2023-02-10 07:55:46,281 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be-bottom 2023-02-10 07:55:46,285 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a-bottom 2023-02-10 07:55:46,288 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63-bottom 2023-02-10 07:55:46,291 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6-bottom 2023-02-10 07:55:46,295 DEBUG [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083-bottom 2023-02-10 07:55:46,295 INFO [StoreOpener-46141c32a6ecc87193e20300b9ff0918-1] regionserver.HStore(310): Store=46141c32a6ecc87193e20300b9ff0918/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:46,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:46,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:46,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 46141c32a6ecc87193e20300b9ff0918; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67053765, jitterRate=-8.21039080619812E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:46,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 46141c32a6ecc87193e20300b9ff0918: Running coprocessor pre-open hook at 1676015746228Writing region info on filesystem at 1676015746228Initializing all the Stores at 1676015746229 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015746229Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015746229Cleaning up temporary data from old regions at 1676015746297 (+68 ms)Cleaning up detritus from prior splits at 1676015746297Running coprocessor post-open hooks at 1676015746299 (+2 ms)Region opened successfully at 1676015746299 2023-02-10 07:55:46,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918., pid=118, masterSystemTime=1676015746225 2023-02-10 07:55:46,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:46,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:55:46,302 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:46,303 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:46,303 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:46,303 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:46,303 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1912): 46141c32a6ecc87193e20300b9ff0918/cf1 is initiating minor compaction (all files) 2023-02-10 07:55:46,303 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1912): 46141c32a6ecc87193e20300b9ff0918/cf2 is initiating minor compaction (all files) 2023-02-10 07:55:46,303 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 46141c32a6ecc87193e20300b9ff0918/cf1 in testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:46,303 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 46141c32a6ecc87193e20300b9ff0918/cf2 in testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:46,304 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/.tmp, totalSize=34.9 K 2023-02-10 07:55:46,304 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/.tmp, totalSize=34.9 K 2023-02-10 07:55:46,304 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015741710 2023-02-10 07:55:46,304 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015741710 2023-02-10 07:55:46,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:46,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:46,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bb5e738408977bf961a4dfb6fec29b21, NAME => 'testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.', STARTKEY => '41', ENDKEY => ''} 2023-02-10 07:55:46,305 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015741875 2023-02-10 07:55:46,305 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015741875 2023-02-10 07:55:46,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=46141c32a6ecc87193e20300b9ff0918, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:46,305 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015746305"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015746305"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015746305"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015746305"}]},"ts":"1676015746305"} 2023-02-10 07:55:46,305 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015742043 2023-02-10 07:55:46,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:46,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,306 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015742043 2023-02-10 07:55:46,306 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015742211 2023-02-10 07:55:46,307 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015742211 2023-02-10 07:55:46,307 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015742407 2023-02-10 07:55:46,307 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015742407 2023-02-10 07:55:46,307 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015742575 2023-02-10 07:55:46,308 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015742575 2023-02-10 07:55:46,308 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015742751 2023-02-10 07:55:46,308 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015742751 2023-02-10 07:55:46,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-10 07:55:46,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 46141c32a6ecc87193e20300b9ff0918, server=jenkins-hbase4.apache.org,43211,1676015645706 in 230 msec 2023-02-10 07:55:46,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, ASSIGN in 391 msec 2023-02-10 07:55:46,313 INFO [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,314 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1 2023-02-10 07:55:46,314 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1 2023-02-10 07:55:46,314 INFO [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb5e738408977bf961a4dfb6fec29b21 columnFamilyName cf1 2023-02-10 07:55:46,321 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] throttle.PressureAwareThroughputController(145): 46141c32a6ecc87193e20300b9ff0918#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:46,326 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] throttle.PressureAwareThroughputController(145): 46141c32a6ecc87193e20300b9ff0918#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:46,331 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588-top 2023-02-10 07:55:46,337 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be-top 2023-02-10 07:55:46,340 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5-top 2023-02-10 07:55:46,342 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/.tmp/cf1/a1f6a19f54d1486db62362231e3f470f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/a1f6a19f54d1486db62362231e3f470f 2023-02-10 07:55:46,344 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/.tmp/cf2/f53322e4198b4f44a71adf0494b2f440 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f53322e4198b4f44a71adf0494b2f440 2023-02-10 07:55:46,344 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091-top 2023-02-10 07:55:46,346 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46141c32a6ecc87193e20300b9ff0918/cf1 of 46141c32a6ecc87193e20300b9ff0918 into a1f6a19f54d1486db62362231e3f470f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:46,346 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 46141c32a6ecc87193e20300b9ff0918: 2023-02-10 07:55:46,346 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918., storeName=46141c32a6ecc87193e20300b9ff0918/cf1, priority=9, startTime=1676015746300; duration=0sec 2023-02-10 07:55:46,346 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:46,348 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219-top 2023-02-10 07:55:46,348 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 46141c32a6ecc87193e20300b9ff0918/cf2 of 46141c32a6ecc87193e20300b9ff0918 into f53322e4198b4f44a71adf0494b2f440(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:46,348 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 46141c32a6ecc87193e20300b9ff0918: 2023-02-10 07:55:46,348 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918., storeName=46141c32a6ecc87193e20300b9ff0918/cf2, priority=9, startTime=1676015746300; duration=0sec 2023-02-10 07:55:46,348 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:46,351 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4-top 2023-02-10 07:55:46,353 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157-top 2023-02-10 07:55:46,353 INFO [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(310): Store=bb5e738408977bf961a4dfb6fec29b21/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:46,354 INFO [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,354 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2 2023-02-10 07:55:46,354 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2 2023-02-10 07:55:46,355 INFO [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb5e738408977bf961a4dfb6fec29b21 columnFamilyName cf2 2023-02-10 07:55:46,365 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa-top 2023-02-10 07:55:46,368 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c-top 2023-02-10 07:55:46,370 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be-top 2023-02-10 07:55:46,373 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a-top 2023-02-10 07:55:46,376 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63-top 2023-02-10 07:55:46,379 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6-top 2023-02-10 07:55:46,384 DEBUG [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083-top 2023-02-10 07:55:46,384 INFO [StoreOpener-bb5e738408977bf961a4dfb6fec29b21-1] regionserver.HStore(310): Store=bb5e738408977bf961a4dfb6fec29b21/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:46,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:46,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bb5e738408977bf961a4dfb6fec29b21; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73983258, jitterRate=0.10243645310401917}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:46,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bb5e738408977bf961a4dfb6fec29b21: Running coprocessor pre-open hook at 1676015746306Writing region info on filesystem at 1676015746306Initializing all the Stores at 1676015746307 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015746307Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015746312 (+5 ms)Cleaning up temporary data from old regions at 1676015746386 (+74 ms)Cleaning up detritus from prior splits at 1676015746387 (+1 ms)Running coprocessor post-open hooks at 1676015746388 (+1 ms)Region opened successfully at 1676015746388 2023-02-10 07:55:46,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21., pid=117, masterSystemTime=1676015746225 2023-02-10 07:55:46,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:46,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:46,390 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:46,390 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:46,390 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,390 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,390 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1912): bb5e738408977bf961a4dfb6fec29b21/cf2 is initiating minor compaction (all files) 2023-02-10 07:55:46,390 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1912): bb5e738408977bf961a4dfb6fec29b21/cf1 is initiating minor compaction (all files) 2023-02-10 07:55:46,391 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bb5e738408977bf961a4dfb6fec29b21/cf2 in testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,391 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bb5e738408977bf961a4dfb6fec29b21/cf1 in testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,391 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa-top] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/.tmp, totalSize=34.9 K 2023-02-10 07:55:46,391 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4-top] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/.tmp, totalSize=34.9 K 2023-02-10 07:55:46,391 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015741710 2023-02-10 07:55:46,392 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015741710 2023-02-10 07:55:46,392 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015741875 2023-02-10 07:55:46,392 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015741875 2023-02-10 07:55:46,392 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015742043 2023-02-10 07:55:46,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,393 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015742043 2023-02-10 07:55:46,393 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=bb5e738408977bf961a4dfb6fec29b21, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,393 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015742211 2023-02-10 07:55:46,393 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015742211 2023-02-10 07:55:46,393 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015746393"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015746393"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015746393"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015746393"}]},"ts":"1676015746393"} 2023-02-10 07:55:46,393 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676015742407 2023-02-10 07:55:46,393 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676015742407 2023-02-10 07:55:46,394 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676015742575 2023-02-10 07:55:46,394 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting 2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676015742575 2023-02-10 07:55:46,394 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] compactions.Compactor(207): Compacting 6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015742751 2023-02-10 07:55:46,394 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] compactions.Compactor(207): Compacting bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015742751 2023-02-10 07:55:46,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-10 07:55:46,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure bb5e738408977bf961a4dfb6fec29b21, server=jenkins-hbase4.apache.org,43211,1676015645706 in 320 msec 2023-02-10 07:55:46,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-10 07:55:46,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, ASSIGN in 479 msec 2023-02-10 07:55:46,398 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-10 07:55:46,398 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:46,398 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-10 07:55:46,404 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] throttle.PressureAwareThroughputController(145): bb5e738408977bf961a4dfb6fec29b21#cf2#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:46,404 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] throttle.PressureAwareThroughputController(145): bb5e738408977bf961a4dfb6fec29b21#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:46,419 INFO [Listener at localhost/43795] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,419 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-10 07:55:46,419 INFO [Listener at localhost/43795] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:46,419 INFO [Listener at localhost/43795] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:46,419 ERROR [Listener at localhost/43795] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-10 07:55:46,419 INFO [Listener at localhost/43795] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-10 07:55:46,419 INFO [Listener at localhost/43795] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-10 07:55:46,419 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-10 07:55:46,420 INFO [Listener at localhost/43795] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-10 07:55:46,420 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-10 07:55:46,422 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-10 07:55:46,422 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-10 07:55:46,422 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-10 07:55:46,422 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 21] 2023-02-10 07:55:46,422 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-02-10 07:55:46,422 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 23] 2023-02-10 07:55:46,422 DEBUG [Listener at localhost/43795] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-02-10 07:55:46,423 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-10 07:55:46,423 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-10 07:55:46,423 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-10 07:55:46,423 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-10 07:55:46,424 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/.tmp/cf2/d6b3e8c919e34f88a1ecad81d763cbb6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/d6b3e8c919e34f88a1ecad81d763cbb6 2023-02-10 07:55:46,424 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:46,424 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/.tmp/cf1/044aaa607513440f955db2a8730a0ae3 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/044aaa607513440f955db2a8730a0ae3 2023-02-10 07:55:46,424 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=1d79b8ea1a81e86b36c3306b6c1de43d, daughterA=817f15842cc8547f342051dd23a15f24, daughterB=a5fd8742bb346a85e0d5f561d7ce5673 2023-02-10 07:55:46,424 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:46,424 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-10 07:55:46,424 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:46,425 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=1aecee45528a1e26504546f287f20ff3, daughterA=3817a509de9c51aaf4f3e11d1b870d42, daughterB=39a95a9bfb795ab2b5aa9c3c4cbfb443 2023-02-10 07:55:46,425 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:46,425 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-10 07:55:46,425 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:46,425 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=324ce5d5ecf6b4389a1c27af6c1de8c6, daughterA=f2991a54330cb9cf607bcfb8f78193ec, daughterB=7a0bbd91d8a69cf2357c5f560a7502d1 2023-02-10 07:55:46,426 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=324ce5d5ecf6b4389a1c27af6c1de8c6, ASSIGN 2023-02-10 07:55:46,426 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:46,426 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-10 07:55:46,426 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:46,426 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=c19064efdfba6693383b2ff7c0db95fd, daughterA=6ed10e80b06bcb9df7beac19c057004b, daughterB=b6f91ff4a65b935935650d5b31c66506 2023-02-10 07:55:46,426 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:46,426 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-10 07:55:46,427 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:46,427 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=09f620010a784524f1f7bb2817b44a9f, daughterA=e55fa0b419778bd958cda69bfff47ba8, daughterB=ecec840173cb92d09b34c16fdbcc87bb 2023-02-10 07:55:46,427 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:46,427 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-10 07:55:46,427 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:46,428 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:46,428 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRollbackForSplitTableRegionWithReplica, region=738761cd8c8642035da172e02506c7fc is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=8e2fb35af8b7ceda6eef617b8936238d, daughterA=cb3130ca3a5218724c2f11bb136c1328, daughterB=6c9e8a50028b6fbf847c5566e6ef2df1 2023-02-10 07:55:46,428 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:46,428 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-10 07:55:46,428 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:46,429 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:46,429 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-10 07:55:46,429 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:46,430 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb5e738408977bf961a4dfb6fec29b21/cf1 of bb5e738408977bf961a4dfb6fec29b21 into 044aaa607513440f955db2a8730a0ae3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:46,430 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb5e738408977bf961a4dfb6fec29b21: 2023-02-10 07:55:46,430 INFO [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21., storeName=bb5e738408977bf961a4dfb6fec29b21/cf1, priority=9, startTime=1676015746389; duration=0sec 2023-02-10 07:55:46,430 DEBUG [RS:1;jenkins-hbase4:43211-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:46,430 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,431 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN 2023-02-10 07:55:46,431 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,431 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, ASSIGN 2023-02-10 07:55:46,431 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, ASSIGN 2023-02-10 07:55:46,431 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure bb5e738408977bf961a4dfb6fec29b21, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,431 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 46141c32a6ecc87193e20300b9ff0918, server=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,431 DEBUG [Listener at localhost/43795] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 held the lock before restarting, call acquireLock to restore it. 2023-02-10 07:55:46,431 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,431 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,431 INFO [Listener at localhost/43795] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,432 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:46,432 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 7ebee4f2e2077f5c617dbf6b2850d8b2, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:46,432 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:46,432 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:46,432 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure bb5e738408977bf961a4dfb6fec29b21, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:46,432 DEBUG [Listener at localhost/43795] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 46141c32a6ecc87193e20300b9ff0918, server=jenkins-hbase4.apache.org,43211,1676015645706 didn't hold the lock before restarting, skip acquiring lock. 2023-02-10 07:55:46,432 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-10 07:55:46,433 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb5e738408977bf961a4dfb6fec29b21/cf2 of bb5e738408977bf961a4dfb6fec29b21 into d6b3e8c919e34f88a1ecad81d763cbb6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:46,433 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb5e738408977bf961a4dfb6fec29b21: 2023-02-10 07:55:46,433 INFO [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21., storeName=bb5e738408977bf961a4dfb6fec29b21/cf2, priority=9, startTime=1676015746390; duration=0sec 2023-02-10 07:55:46,433 DEBUG [RS:1;jenkins-hbase4:43211-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:46,434 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-10 07:55:46,435 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736, table=hbase:meta, region=1588230740 2023-02-10 07:55:46,435 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676015776435 2023-02-10 07:55:46,437 DEBUG [Listener at localhost/43795] assignment.AssignmentManager(1519): Joining cluster... 2023-02-10 07:55:46,437 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,439 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=a50233cec7373b84b61f6ef56d6ed02c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,40867,1676015645736, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736, openSeqNum=2 2023-02-10 07:55:46,439 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=7ebee4f2e2077f5c617dbf6b2850d8b2, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=2 2023-02-10 07:55:46,439 INFO [Listener at localhost/43795] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,43211,1676015645706, table=testRecoveryAndDoubleExecution, region=7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,439 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=46141c32a6ecc87193e20300b9ff0918, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=86 2023-02-10 07:55:46,440 DEBUG [Listener at localhost/43795] assignment.RegionStateStore(147): Load hbase:meta entry region=bb5e738408977bf961a4dfb6fec29b21, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,43211,1676015645706, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706, openSeqNum=86 2023-02-10 07:55:46,440 INFO [Listener at localhost/43795] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-10 07:55:46,440 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676015806440 2023-02-10 07:55:46,440 INFO [Listener at localhost/43795] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676015866440 2023-02-10 07:55:46,440 INFO [Listener at localhost/43795] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-10 07:55:46,440 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-10 07:55:46,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=7ebee4f2e2077f5c617dbf6b2850d8b2, daughterA=46141c32a6ecc87193e20300b9ff0918, daughterB=bb5e738408977bf961a4dfb6fec29b21 in 3.4290 sec 2023-02-10 07:55:46,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43211] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:58646 deadline: 1676015806691, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2. is not online on jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,802 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-10 07:55:46,807 INFO [Listener at localhost/43795] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-10 07:55:46,808 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-10 07:55:46,820 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:46,821 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-10 07:55:46,821 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-10 07:55:46,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:46,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-10 07:55:46,824 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015746824"}]},"ts":"1676015746824"} 2023-02-10 07:55:46,825 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-10 07:55:46,827 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-10 07:55:46,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, UNASSIGN}] 2023-02-10 07:55:46,829 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, UNASSIGN 2023-02-10 07:55:46,829 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=bb5e738408977bf961a4dfb6fec29b21, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,830 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015746829"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015746829"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015746829"}]},"ts":"1676015746829"} 2023-02-10 07:55:46,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure bb5e738408977bf961a4dfb6fec29b21, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:55:46,833 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, UNASSIGN 2023-02-10 07:55:46,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=46141c32a6ecc87193e20300b9ff0918, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:46,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015746833"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015746833"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015746833"}]},"ts":"1676015746833"} 2023-02-10 07:55:46,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 46141c32a6ecc87193e20300b9ff0918, server=jenkins-hbase4.apache.org,43211,1676015645706}] 2023-02-10 07:55:46,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-10 07:55:46,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:46,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bb5e738408977bf961a4dfb6fec29b21, disabling compactions & flushes 2023-02-10 07:55:46,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. after waiting 0 ms 2023-02-10 07:55:46,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:46,983 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4-top] to archive 2023-02-10 07:55:46,984 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:46,985 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,986 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,987 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,988 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,989 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,990 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,991 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,992 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa-top] to archive 2023-02-10 07:55:46,993 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:46,995 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,996 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,996 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,997 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,998 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:46,999 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,000 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,003 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:55:47,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:47,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21. 2023-02-10 07:55:47,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bb5e738408977bf961a4dfb6fec29b21: Waiting for close lock at 1676015746982Running coprocessor pre-close hooks at 1676015746982Disabling compacts and flushes for region at 1676015746982Disabling writes for close at 1676015746982Writing region close event to WAL at 1676015747001 (+19 ms)Running coprocessor post-close hooks at 1676015747004 (+3 ms)Closed at 1676015747004 2023-02-10 07:55:47,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:47,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:47,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 46141c32a6ecc87193e20300b9ff0918, disabling compactions & flushes 2023-02-10 07:55:47,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:47,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:47,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. after waiting 0 ms 2023-02-10 07:55:47,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:47,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=bb5e738408977bf961a4dfb6fec29b21, regionState=CLOSED 2023-02-10 07:55:47,007 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015747006"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015747006"}]},"ts":"1676015747006"} 2023-02-10 07:55:47,007 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4-bottom] to archive 2023-02-10 07:55:47,008 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:47,009 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-10 07:55:47,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure bb5e738408977bf961a4dfb6fec29b21, server=jenkins-hbase4.apache.org,43211,1676015645706 in 177 msec 2023-02-10 07:55:47,010 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bb5e738408977bf961a4dfb6fec29b21, UNASSIGN in 182 msec 2023-02-10 07:55:47,011 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,012 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,013 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,014 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,015 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,016 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa-bottom] to archive 2023-02-10 07:55:47,017 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:47,018 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,019 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,020 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,021 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,022 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,023 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,023 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:55:47,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:47,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918. 2023-02-10 07:55:47,027 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 46141c32a6ecc87193e20300b9ff0918: Waiting for close lock at 1676015747006Running coprocessor pre-close hooks at 1676015747006Disabling compacts and flushes for region at 1676015747006Disabling writes for close at 1676015747006Writing region close event to WAL at 1676015747024 (+18 ms)Running coprocessor post-close hooks at 1676015747027 (+3 ms)Closed at 1676015747027 2023-02-10 07:55:47,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:47,029 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=46141c32a6ecc87193e20300b9ff0918, regionState=CLOSED 2023-02-10 07:55:47,029 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676015747029"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015747029"}]},"ts":"1676015747029"} 2023-02-10 07:55:47,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-10 07:55:47,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 46141c32a6ecc87193e20300b9ff0918, server=jenkins-hbase4.apache.org,43211,1676015645706 in 196 msec 2023-02-10 07:55:47,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-10 07:55:47,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=46141c32a6ecc87193e20300b9ff0918, UNASSIGN in 203 msec 2023-02-10 07:55:47,033 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015747033"}]},"ts":"1676015747033"} 2023-02-10 07:55:47,034 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-10 07:55:47,036 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-10 07:55:47,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 214 msec 2023-02-10 07:55:47,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-10 07:55:47,126 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-10 07:55:47,127 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-10 07:55:47,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:47,129 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:47,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-10 07:55:47,130 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:47,134 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,134 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:47,134 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:47,136 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/recovered.edits] 2023-02-10 07:55:47,136 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/recovered.edits] 2023-02-10 07:55:47,136 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/recovered.edits] 2023-02-10 07:55:47,139 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/.regioninfo to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/.regioninfo 2023-02-10 07:55:47,139 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/.regioninfo to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/.regioninfo 2023-02-10 07:55:47,139 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/2763456ca0234e6fa0fcb64fd97a2588 2023-02-10 07:55:47,141 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/4922a75d9b234f70934ff635476918be 2023-02-10 07:55:47,142 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/797ff7b0ceab45c483921d3caddb26a5 2023-02-10 07:55:47,142 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,142 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/2763456ca0234e6fa0fcb64fd97a2588.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,143 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/870fddc159ba4cdea847fe4fb45b6091 2023-02-10 07:55:47,143 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,143 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/4922a75d9b234f70934ff635476918be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,144 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bc5ca2d78f694c79b2acd92484a87219 2023-02-10 07:55:47,145 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,145 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/797ff7b0ceab45c483921d3caddb26a5.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,146 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/bedaaf6912494fa68f9787fc7dd5afd4 2023-02-10 07:55:47,146 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,146 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/870fddc159ba4cdea847fe4fb45b6091.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,147 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf1/d68310def3744112b9630c3f0ab1a157 2023-02-10 07:55:47,147 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,147 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/bc5ca2d78f694c79b2acd92484a87219.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,148 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,148 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/bedaaf6912494fa68f9787fc7dd5afd4.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,149 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,150 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/6b34b649159d4f06bf67b6523534c2aa 2023-02-10 07:55:47,150 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf1/d68310def3744112b9630c3f0ab1a157.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,151 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/84741a528c85491ca9ab65911b91bb6c 2023-02-10 07:55:47,152 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be 2023-02-10 07:55:47,152 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,153 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/6b34b649159d4f06bf67b6523534c2aa.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,153 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/8abfcaea20f3486ca0e9ffca5ed2378a 2023-02-10 07:55:47,153 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,154 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/84741a528c85491ca9ab65911b91bb6c.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,154 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/9cde6baa229c4d918d51e46c54c56e63 2023-02-10 07:55:47,155 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,155 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/8ab1fa81cfd545caaf6ee8f3ab3676be.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,156 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/b60e10b0dad546258c63c6de371154a6 2023-02-10 07:55:47,156 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,156 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/8abfcaea20f3486ca0e9ffca5ed2378a.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,157 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083 2023-02-10 07:55:47,157 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,158 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/9cde6baa229c4d918d51e46c54c56e63.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,158 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,159 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/b60e10b0dad546258c63c6de371154a6.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,160 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/46141c32a6ecc87193e20300b9ff0918/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,160 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/recovered.edits/85.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2/recovered.edits/85.seqid 2023-02-10 07:55:47,161 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,161 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/bb5e738408977bf961a4dfb6fec29b21/cf2/f7fcbbfcdf5b47a19ccf12666bbb6083.7ebee4f2e2077f5c617dbf6b2850d8b2 2023-02-10 07:55:47,162 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/a1f6a19f54d1486db62362231e3f470f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf1/a1f6a19f54d1486db62362231e3f470f 2023-02-10 07:55:47,163 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/044aaa607513440f955db2a8730a0ae3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf1/044aaa607513440f955db2a8730a0ae3 2023-02-10 07:55:47,164 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f53322e4198b4f44a71adf0494b2f440 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/cf2/f53322e4198b4f44a71adf0494b2f440 2023-02-10 07:55:47,165 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/d6b3e8c919e34f88a1ecad81d763cbb6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/cf2/d6b3e8c919e34f88a1ecad81d763cbb6 2023-02-10 07:55:47,166 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918/recovered.edits/90.seqid 2023-02-10 07:55:47,167 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/46141c32a6ecc87193e20300b9ff0918 2023-02-10 07:55:47,167 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21/recovered.edits/90.seqid 2023-02-10 07:55:47,168 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testRecoveryAndDoubleExecution/bb5e738408977bf961a4dfb6fec29b21 2023-02-10 07:55:47,168 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-10 07:55:47,169 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:47,171 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-10 07:55:47,172 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-10 07:55:47,173 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:47,173 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-10 07:55:47,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015747173"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:47,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015747173"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:47,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015747173"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:47,175 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-10 07:55:47,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7ebee4f2e2077f5c617dbf6b2850d8b2, NAME => 'testRecoveryAndDoubleExecution,,1676015741207.7ebee4f2e2077f5c617dbf6b2850d8b2.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 46141c32a6ecc87193e20300b9ff0918, NAME => 'testRecoveryAndDoubleExecution,,1676015743011.46141c32a6ecc87193e20300b9ff0918.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => bb5e738408977bf961a4dfb6fec29b21, NAME => 'testRecoveryAndDoubleExecution,41,1676015743011.bb5e738408977bf961a4dfb6fec29b21.', STARTKEY => '41', ENDKEY => ''}] 2023-02-10 07:55:47,175 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-10 07:55:47,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015747175"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:47,176 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-10 07:55:47,178 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-10 07:55:47,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 51 msec 2023-02-10 07:55:47,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-10 07:55:47,231 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-10 07:55:47,245 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=378 (was 367) - Thread LEAK? -, OpenFileDescriptor=829 (was 740) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=87 (was 95), ProcessCount=168 (was 168), AvailableMemoryMB=3155 (was 3208) 2023-02-10 07:55:47,260 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=378, OpenFileDescriptor=829, MaxFileDescriptor=60000, SystemLoadAverage=87, ProcessCount=168, AvailableMemoryMB=3154 2023-02-10 07:55:47,261 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:47,264 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:55:47,264 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:55:47,266 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,267 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:55:47,268 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:55:47,270 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,270 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5 empty. 2023-02-10 07:55:47,271 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,271 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-10 07:55:47,279 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-10 07:55:47,280 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9b2eec727609515e00693d8091cd3cd5, NAME => 'testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:55:47,286 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:47,286 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 9b2eec727609515e00693d8091cd3cd5, disabling compactions & flushes 2023-02-10 07:55:47,286 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,286 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,286 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. after waiting 0 ms 2023-02-10 07:55:47,286 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,286 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,287 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 9b2eec727609515e00693d8091cd3cd5: Waiting for close lock at 1676015747286Disabling compacts and flushes for region at 1676015747286Disabling writes for close at 1676015747286Writing region close event to WAL at 1676015747286Closed at 1676015747286 2023-02-10 07:55:47,288 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:55:47,289 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015747289"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015747289"}]},"ts":"1676015747289"} 2023-02-10 07:55:47,290 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:55:47,290 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:55:47,291 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015747290"}]},"ts":"1676015747290"} 2023-02-10 07:55:47,291 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-10 07:55:47,294 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:55:47,294 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:55:47,294 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:55:47,294 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:55:47,295 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:55:47,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b2eec727609515e00693d8091cd3cd5, ASSIGN}] 2023-02-10 07:55:47,296 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b2eec727609515e00693d8091cd3cd5, ASSIGN 2023-02-10 07:55:47,296 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b2eec727609515e00693d8091cd3cd5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:55:47,446 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:55:47,448 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=9b2eec727609515e00693d8091cd3cd5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,448 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015747447"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015747447"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015747447"}]},"ts":"1676015747447"} 2023-02-10 07:55:47,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 9b2eec727609515e00693d8091cd3cd5, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:47,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9b2eec727609515e00693d8091cd3cd5, NAME => 'testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:55:47,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:47,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:47,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,604 INFO [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,605 DEBUG [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1 2023-02-10 07:55:47,605 DEBUG [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1 2023-02-10 07:55:47,606 INFO [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b2eec727609515e00693d8091cd3cd5 columnFamilyName cf1 2023-02-10 07:55:47,606 INFO [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] regionserver.HStore(310): Store=9b2eec727609515e00693d8091cd3cd5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:47,606 INFO [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,607 DEBUG [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2 2023-02-10 07:55:47,607 DEBUG [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2 2023-02-10 07:55:47,607 INFO [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b2eec727609515e00693d8091cd3cd5 columnFamilyName cf2 2023-02-10 07:55:47,608 INFO [StoreOpener-9b2eec727609515e00693d8091cd3cd5-1] regionserver.HStore(310): Store=9b2eec727609515e00693d8091cd3cd5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:47,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:47,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:47,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:55:47,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9b2eec727609515e00693d8091cd3cd5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73820176, jitterRate=0.1000063419342041}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:47,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9b2eec727609515e00693d8091cd3cd5: Running coprocessor pre-open hook at 1676015747603Writing region info on filesystem at 1676015747603Initializing all the Stores at 1676015747604 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015747604Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015747604Cleaning up temporary data from old regions at 1676015747609 (+5 ms)Cleaning up detritus from prior splits at 1676015747609Running coprocessor post-open hooks at 1676015747612 (+3 ms)Region opened successfully at 1676015747612 2023-02-10 07:55:47,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5., pid=127, masterSystemTime=1676015747600 2023-02-10 07:55:47,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=9b2eec727609515e00693d8091cd3cd5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,615 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015747614"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015747614"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015747614"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015747614"}]},"ts":"1676015747614"} 2023-02-10 07:55:47,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-10 07:55:47,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 9b2eec727609515e00693d8091cd3cd5, server=jenkins-hbase4.apache.org,40867,1676015645736 in 167 msec 2023-02-10 07:55:47,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-10 07:55:47,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b2eec727609515e00693d8091cd3cd5, ASSIGN in 322 msec 2023-02-10 07:55:47,619 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:55:47,619 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015747619"}]},"ts":"1676015747619"} 2023-02-10 07:55:47,620 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-10 07:55:47,621 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:55:47,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 357 msec 2023-02-10 07:55:47,772 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:47,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,774 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-10 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-10 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,774 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,775 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,775 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,776 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,776 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:47,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:47,777 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:47,777 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,778 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. started... 2023-02-10 07:55:47,779 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b2eec727609515e00693d8091cd3cd5 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:55:47,779 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,781 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:47,787 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/8e60fd11fb7544bfaf077ec61e61079e 2023-02-10 07:55:47,797 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/0e92ec3fac1d4d689a8ca3b637c351b4 2023-02-10 07:55:47,801 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/8e60fd11fb7544bfaf077ec61e61079e as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/8e60fd11fb7544bfaf077ec61e61079e 2023-02-10 07:55:47,804 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/8e60fd11fb7544bfaf077ec61e61079e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:47,804 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/0e92ec3fac1d4d689a8ca3b637c351b4 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0e92ec3fac1d4d689a8ca3b637c351b4 2023-02-10 07:55:47,807 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0e92ec3fac1d4d689a8ca3b637c351b4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:47,808 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 30ms, sequenceid=6, compaction requested=false 2023-02-10 07:55:47,808 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:47,809 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b2eec727609515e00693d8091cd3cd5: 2023-02-10 07:55:47,809 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,809 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:47,809 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:47,809 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,809 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:47,809 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:47,810 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,810 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:47,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:47,810 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,810 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:47,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:47,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:47,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:47,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-10 07:55:47,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41c25c15[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:47,812 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-10 07:55:47,812 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:47,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:47,813 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,813 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:47,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:47,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,813 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:47,813 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:47,813 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:47,814 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:47,813 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:47,814 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:47,815 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,815 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:47,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:47,815 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:47,815 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,816 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:47,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:47,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:47,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:55:47,819 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-10 07:55:47,819 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:47,819 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:47,819 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,819 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:47,820 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,820 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:47,821 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,820 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,821 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:47,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,822 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:47,823 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,823 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,824 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,829 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,829 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:47,830 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:47,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,829 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:47,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,830 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,930 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:47,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:47,940 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:47,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:47,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,942 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:47,942 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,942 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-10 07:55:47,942 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-10 07:55:47,942 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,942 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,943 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,943 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,943 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,943 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,944 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,944 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,945 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,946 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:47,946 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:47,946 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:47,946 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:47,946 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:47,946 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,946 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. started... 2023-02-10 07:55:47,946 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b2eec727609515e00693d8091cd3cd5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:47,947 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:47,957 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/20391d5060c04593b0726c428467b759 2023-02-10 07:55:47,969 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/949ab233837749f3a51fb15b1b8dcc45 2023-02-10 07:55:47,972 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/20391d5060c04593b0726c428467b759 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/20391d5060c04593b0726c428467b759 2023-02-10 07:55:47,975 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/20391d5060c04593b0726c428467b759, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:47,976 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/949ab233837749f3a51fb15b1b8dcc45 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/949ab233837749f3a51fb15b1b8dcc45 2023-02-10 07:55:47,979 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/949ab233837749f3a51fb15b1b8dcc45, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:47,979 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 33ms, sequenceid=19, compaction requested=false 2023-02-10 07:55:47,979 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b2eec727609515e00693d8091cd3cd5: 2023-02-10 07:55:47,979 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:47,979 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:47,980 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:47,980 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,980 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:47,980 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:47,981 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,981 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:47,981 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,981 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:47,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-10 07:55:47,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c1e5610[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:47,983 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-10 07:55:47,983 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,984 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:47,984 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,986 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:47,986 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:47,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:47,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:55:47,989 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-10 07:55:47,989 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:47,989 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:47,989 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,989 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:47,991 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,991 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:47,991 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,991 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,991 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,992 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,992 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,992 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:47,993 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,993 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,994 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:47,994 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,995 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,997 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,997 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:47,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:47,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:47,997 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:47,997 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:47,998 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:47,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:47,998 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:47,998 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:47,998 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:47,998 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:47,998 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:47,998 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,098 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:48,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:48,110 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:48,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:48,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,112 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,112 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,112 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-10 07:55:48,112 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-10 07:55:48,112 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,112 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,113 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,113 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,113 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,113 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,114 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,114 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:48,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,115 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:48,116 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,116 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. started... 2023-02-10 07:55:48,116 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b2eec727609515e00693d8091cd3cd5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:48,117 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,119 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,130 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/b80c2b11aab748848824b0d498bfe026 2023-02-10 07:55:48,140 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/0da27882ec9747549f75504e28117a4f 2023-02-10 07:55:48,144 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/b80c2b11aab748848824b0d498bfe026 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/b80c2b11aab748848824b0d498bfe026 2023-02-10 07:55:48,146 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/b80c2b11aab748848824b0d498bfe026, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:48,147 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/0da27882ec9747549f75504e28117a4f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0da27882ec9747549f75504e28117a4f 2023-02-10 07:55:48,150 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0da27882ec9747549f75504e28117a4f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:48,151 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 35ms, sequenceid=32, compaction requested=true 2023-02-10 07:55:48,151 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b2eec727609515e00693d8091cd3cd5: 2023-02-10 07:55:48,151 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,151 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:48,151 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:48,151 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,151 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,151 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,152 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,152 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,153 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,153 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-10 07:55:48,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fbbf9a0[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:48,154 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-10 07:55:48,155 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,156 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:48,166 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,166 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:48,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:55:48,169 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-10 07:55:48,169 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:48,169 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:48,169 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,169 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:48,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,171 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,172 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,172 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,174 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,174 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,174 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,177 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,177 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:48,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,177 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,177 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,178 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,278 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:48,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:48,289 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:48,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:48,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,290 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,290 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,290 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-10 07:55:48,290 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-10 07:55:48,291 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,291 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,293 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,293 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,293 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,293 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,293 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,293 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:48,295 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:48,296 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,296 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. started... 2023-02-10 07:55:48,296 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b2eec727609515e00693d8091cd3cd5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:48,296 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,297 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,298 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,298 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,305 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/dd6282a2ab0d42cd975a2449212ff1a3 2023-02-10 07:55:48,315 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/518cf2552ce64ca59c8b63bcfde22c4f 2023-02-10 07:55:48,318 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/dd6282a2ab0d42cd975a2449212ff1a3 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/dd6282a2ab0d42cd975a2449212ff1a3 2023-02-10 07:55:48,321 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/dd6282a2ab0d42cd975a2449212ff1a3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:48,321 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/518cf2552ce64ca59c8b63bcfde22c4f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/518cf2552ce64ca59c8b63bcfde22c4f 2023-02-10 07:55:48,324 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/518cf2552ce64ca59c8b63bcfde22c4f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:48,325 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 29ms, sequenceid=45, compaction requested=true 2023-02-10 07:55:48,325 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b2eec727609515e00693d8091cd3cd5: 2023-02-10 07:55:48,325 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,325 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,326 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,326 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,327 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,327 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-10 07:55:48,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7db43f72[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:48,328 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-10 07:55:48,328 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,330 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,330 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,330 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,331 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,331 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:48,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:48,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:48,331 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:48,332 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,332 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,332 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,333 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,333 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,333 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,333 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,333 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,333 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,333 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:48,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:55:48,336 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-10 07:55:48,336 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:48,336 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:48,336 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,336 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:48,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,337 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,337 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,338 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,338 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,340 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,340 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,340 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,346 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,346 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,346 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:48,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:48,346 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,347 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,354 ERROR [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1326607998-172.31.14.131-1676015641262 (Datanode Uuid e71256cb-90c7-4766-bfdc-18ad52545fac) service to localhost/127.0.0.1:34635 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-10 07:55:48,447 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:48,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:48,458 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:48,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:48,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,459 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-10 07:55:48,459 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-10 07:55:48,460 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,460 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,461 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,461 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,461 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,461 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,461 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,461 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,462 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,462 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:48,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:48,463 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,463 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. started... 2023-02-10 07:55:48,463 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b2eec727609515e00693d8091cd3cd5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,464 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,465 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,471 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/94bbe5706d004456aa6e219189c9d8aa 2023-02-10 07:55:48,480 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/f8af2e2dd5c743fdb84946bab5705268 2023-02-10 07:55:48,483 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/94bbe5706d004456aa6e219189c9d8aa as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/94bbe5706d004456aa6e219189c9d8aa 2023-02-10 07:55:48,487 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/94bbe5706d004456aa6e219189c9d8aa, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:48,487 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/f8af2e2dd5c743fdb84946bab5705268 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/f8af2e2dd5c743fdb84946bab5705268 2023-02-10 07:55:48,490 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/f8af2e2dd5c743fdb84946bab5705268, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:48,491 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 28ms, sequenceid=58, compaction requested=true 2023-02-10 07:55:48,491 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b2eec727609515e00693d8091cd3cd5: 2023-02-10 07:55:48,491 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,491 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:48,491 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:48,491 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,491 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,491 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,498 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,498 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,499 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,499 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-10 07:55:48,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ec5f625[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:48,501 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-10 07:55:48,501 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,502 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:48,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:48,502 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,504 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,505 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,505 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:48,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:55:48,508 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-10 07:55:48,508 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:48,508 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:48,508 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,508 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:48,510 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,510 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,511 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,512 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,512 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,514 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,514 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,514 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,517 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,517 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:48,517 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,518 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,618 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:48,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:48,631 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:48,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:48,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,633 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,633 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,633 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-10 07:55:48,633 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-10 07:55:48,633 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,633 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,635 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,635 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,635 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,635 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,635 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,636 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-10 07:55:48,637 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,637 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. started... 2023-02-10 07:55:48,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,638 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9b2eec727609515e00693d8091cd3cd5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:48,638 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,639 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,639 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,640 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,658 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/ec9294852b24438aa60718b02430f84f 2023-02-10 07:55:48,670 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/09406c0b35b3418da405f0c4c5745dcf 2023-02-10 07:55:48,673 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/ec9294852b24438aa60718b02430f84f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/ec9294852b24438aa60718b02430f84f 2023-02-10 07:55:48,676 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/ec9294852b24438aa60718b02430f84f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:48,677 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/09406c0b35b3418da405f0c4c5745dcf as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/09406c0b35b3418da405f0c4c5745dcf 2023-02-10 07:55:48,680 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/09406c0b35b3418da405f0c4c5745dcf, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:48,680 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 42ms, sequenceid=71, compaction requested=true 2023-02-10 07:55:48,680 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9b2eec727609515e00693d8091cd3cd5: 2023-02-10 07:55:48,680 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,681 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:48,681 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:48,681 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,681 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-10 07:55:48,681 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-10 07:55:48,682 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,682 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,682 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,682 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:48,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-10 07:55:48,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50159b3b[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:48,684 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-10 07:55:48,684 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,687 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,687 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,687 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:48,688 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,688 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:48,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-10 07:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:48,690 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,690 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-10 07:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:48,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-10 07:55:48,694 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-10 07:55:48,694 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:48,694 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:48,694 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,694 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:48,695 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,695 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,695 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,695 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,696 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:48,696 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,695 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:48,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,696 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,697 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:48,697 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:48,698 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,699 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,699 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-10 07:55:48,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:48,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,701 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:48,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,701 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:48,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:48,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:48,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,702 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-10 07:55:48,802 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:48,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:48,810 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,811 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=9b2eec727609515e00693d8091cd3cd5, daughterA=8570082a7f95a7fb8bb41d51fac7f3ee, daughterB=0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:48,812 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=9b2eec727609515e00693d8091cd3cd5, daughterA=8570082a7f95a7fb8bb41d51fac7f3ee, daughterB=0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:48,812 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=9b2eec727609515e00693d8091cd3cd5, daughterA=8570082a7f95a7fb8bb41d51fac7f3ee, daughterB=0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:48,812 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=9b2eec727609515e00693d8091cd3cd5, daughterA=8570082a7f95a7fb8bb41d51fac7f3ee, daughterB=0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:48,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b2eec727609515e00693d8091cd3cd5, UNASSIGN}] 2023-02-10 07:55:48,814 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b2eec727609515e00693d8091cd3cd5, UNASSIGN 2023-02-10 07:55:48,814 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=9b2eec727609515e00693d8091cd3cd5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:48,814 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015748814"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015748814"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015748814"}]},"ts":"1676015748814"} 2023-02-10 07:55:48,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 9b2eec727609515e00693d8091cd3cd5, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:48,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:48,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9b2eec727609515e00693d8091cd3cd5, disabling compactions & flushes 2023-02-10 07:55:48,967 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. after waiting 0 ms 2023-02-10 07:55:48,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:48,967 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 9b2eec727609515e00693d8091cd3cd5 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-10 07:55:48,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/07e4d2212bc74b16956a4e9e57a671b6 2023-02-10 07:55:48,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/6cefa5af3b484804a21c3f2b1697d809 2023-02-10 07:55:48,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf1/07e4d2212bc74b16956a4e9e57a671b6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/07e4d2212bc74b16956a4e9e57a671b6 2023-02-10 07:55:48,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/07e4d2212bc74b16956a4e9e57a671b6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:48,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.tmp/cf2/6cefa5af3b484804a21c3f2b1697d809 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/6cefa5af3b484804a21c3f2b1697d809 2023-02-10 07:55:49,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/6cefa5af3b484804a21c3f2b1697d809, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:49,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 33ms, sequenceid=82, compaction requested=true 2023-02-10 07:55:49,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-10 07:55:49,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:49,005 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. 2023-02-10 07:55:49,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9b2eec727609515e00693d8091cd3cd5: Waiting for close lock at 1676015748967Running coprocessor pre-close hooks at 1676015748967Disabling compacts and flushes for region at 1676015748967Disabling writes for close at 1676015748967Obtaining lock to block concurrent updates at 1676015748967Preparing flush snapshotting stores in 9b2eec727609515e00693d8091cd3cd5 at 1676015748967Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676015748967Flushing stores of testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. at 1676015748967Flushing 9b2eec727609515e00693d8091cd3cd5/cf1: creating writer at 1676015748970 (+3 ms)Flushing 9b2eec727609515e00693d8091cd3cd5/cf1: appending metadata at 1676015748971 (+1 ms)Flushing 9b2eec727609515e00693d8091cd3cd5/cf1: closing flushed file at 1676015748971Flushing 9b2eec727609515e00693d8091cd3cd5/cf2: creating writer at 1676015748983 (+12 ms)Flushing 9b2eec727609515e00693d8091cd3cd5/cf2: appending metadata at 1676015748985 (+2 ms)Flushing 9b2eec727609515e00693d8091cd3cd5/cf2: closing flushed file at 1676015748985Flushing 9b2eec727609515e00693d8091cd3cd5/cf1: reopening flushed file at 1676015748993 (+8 ms)Flushing 9b2eec727609515e00693d8091cd3cd5/cf2: reopening flushed file at 1676015748997 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 9b2eec727609515e00693d8091cd3cd5 in 33ms, sequenceid=82, compaction requested=true at 1676015749000 (+3 ms)Writing region close event to WAL at 1676015749002 (+2 ms)Running coprocessor post-close hooks at 1676015749005 (+3 ms)Closed at 1676015749005 2023-02-10 07:55:49,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=9b2eec727609515e00693d8091cd3cd5, regionState=CLOSED 2023-02-10 07:55:49,007 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015749006"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015749006"}]},"ts":"1676015749006"} 2023-02-10 07:55:49,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-10 07:55:49,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 9b2eec727609515e00693d8091cd3cd5, server=jenkins-hbase4.apache.org,40867,1676015645736 in 193 msec 2023-02-10 07:55:49,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-10 07:55:49,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=9b2eec727609515e00693d8091cd3cd5, UNASSIGN in 196 msec 2023-02-10 07:55:49,021 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=9b2eec727609515e00693d8091cd3cd5, threads=14 2023-02-10 07:55:49,022 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/09406c0b35b3418da405f0c4c5745dcf for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,022 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0da27882ec9747549f75504e28117a4f for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,022 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0e92ec3fac1d4d689a8ca3b637c351b4 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,022 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/518cf2552ce64ca59c8b63bcfde22c4f for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,024 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/6cefa5af3b484804a21c3f2b1697d809 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,024 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/949ab233837749f3a51fb15b1b8dcc45 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,024 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/f8af2e2dd5c743fdb84946bab5705268 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,024 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/07e4d2212bc74b16956a4e9e57a671b6 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,030 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/20391d5060c04593b0726c428467b759 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,030 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/8e60fd11fb7544bfaf077ec61e61079e for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,031 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/94bbe5706d004456aa6e219189c9d8aa for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,031 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/dd6282a2ab0d42cd975a2449212ff1a3 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,031 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/b80c2b11aab748848824b0d498bfe026 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,031 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/ec9294852b24438aa60718b02430f84f for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,038 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/6cefa5af3b484804a21c3f2b1697d809, top=false 2023-02-10 07:55:49,038 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0e92ec3fac1d4d689a8ca3b637c351b4, top=false 2023-02-10 07:55:49,038 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/09406c0b35b3418da405f0c4c5745dcf, top=false 2023-02-10 07:55:49,038 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0da27882ec9747549f75504e28117a4f, top=false 2023-02-10 07:55:49,039 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/949ab233837749f3a51fb15b1b8dcc45, top=false 2023-02-10 07:55:49,041 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/f8af2e2dd5c743fdb84946bab5705268, top=false 2023-02-10 07:55:49,042 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/07e4d2212bc74b16956a4e9e57a671b6, top=false 2023-02-10 07:55:49,042 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/8e60fd11fb7544bfaf077ec61e61079e, top=false 2023-02-10 07:55:49,043 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/dd6282a2ab0d42cd975a2449212ff1a3, top=false 2023-02-10 07:55:49,043 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/b80c2b11aab748848824b0d498bfe026, top=false 2023-02-10 07:55:49,044 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/94bbe5706d004456aa6e219189c9d8aa, top=false 2023-02-10 07:55:49,045 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/518cf2552ce64ca59c8b63bcfde22c4f, top=false 2023-02-10 07:55:49,047 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/20391d5060c04593b0726c428467b759, top=false 2023-02-10 07:55:49,048 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/ec9294852b24438aa60718b02430f84f, top=false 2023-02-10 07:55:49,108 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-518cf2552ce64ca59c8b63bcfde22c4f for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,112 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/518cf2552ce64ca59c8b63bcfde22c4f for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,140 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0e92ec3fac1d4d689a8ca3b637c351b4 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,140 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-dd6282a2ab0d42cd975a2449212ff1a3 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,140 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-f8af2e2dd5c743fdb84946bab5705268 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,140 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-07e4d2212bc74b16956a4e9e57a671b6 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,145 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/07e4d2212bc74b16956a4e9e57a671b6 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,146 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/dd6282a2ab0d42cd975a2449212ff1a3 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,146 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0e92ec3fac1d4d689a8ca3b637c351b4 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,146 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/f8af2e2dd5c743fdb84946bab5705268 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,152 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-949ab233837749f3a51fb15b1b8dcc45 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,152 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-ec9294852b24438aa60718b02430f84f for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,156 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/ec9294852b24438aa60718b02430f84f for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,157 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/949ab233837749f3a51fb15b1b8dcc45 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,160 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-6cefa5af3b484804a21c3f2b1697d809 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,162 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-09406c0b35b3418da405f0c4c5745dcf for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,162 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-b80c2b11aab748848824b0d498bfe026 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,164 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0da27882ec9747549f75504e28117a4f for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,164 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/6cefa5af3b484804a21c3f2b1697d809 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,165 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-94bbe5706d004456aa6e219189c9d8aa for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,165 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-20391d5060c04593b0726c428467b759 for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,165 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/.splits/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-8e60fd11fb7544bfaf077ec61e61079e for child: 8570082a7f95a7fb8bb41d51fac7f3ee, parent: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,167 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/b80c2b11aab748848824b0d498bfe026 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,169 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/09406c0b35b3418da405f0c4c5745dcf for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,169 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0da27882ec9747549f75504e28117a4f for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,170 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/8e60fd11fb7544bfaf077ec61e61079e for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,171 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/94bbe5706d004456aa6e219189c9d8aa for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,171 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/20391d5060c04593b0726c428467b759 for region: 9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:49,171 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 9b2eec727609515e00693d8091cd3cd5 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-10 07:55:49,205 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:55:49,206 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:55:49,208 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676015749207"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676015749207"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676015749207"}]},"ts":"1676015749207"} 2023-02-10 07:55:49,208 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749207"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015749207"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015749207"}]},"ts":"1676015749207"} 2023-02-10 07:55:49,208 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749207"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015749207"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015749207"}]},"ts":"1676015749207"} 2023-02-10 07:55:49,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8570082a7f95a7fb8bb41d51fac7f3ee, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0da42b6033eb09e7c27060df73d4da34, ASSIGN}] 2023-02-10 07:55:49,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0da42b6033eb09e7c27060df73d4da34, ASSIGN 2023-02-10 07:55:49,212 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0da42b6033eb09e7c27060df73d4da34, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:55:49,212 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8570082a7f95a7fb8bb41d51fac7f3ee, ASSIGN 2023-02-10 07:55:49,213 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8570082a7f95a7fb8bb41d51fac7f3ee, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:55:49,362 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-10 07:55:49,364 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=8570082a7f95a7fb8bb41d51fac7f3ee, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:49,364 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749364"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015749364"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015749364"}]},"ts":"1676015749364"} 2023-02-10 07:55:49,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure 8570082a7f95a7fb8bb41d51fac7f3ee, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:49,368 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=0da42b6033eb09e7c27060df73d4da34, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:49,368 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749368"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015749368"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015749368"}]},"ts":"1676015749368"} 2023-02-10 07:55:49,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure 0da42b6033eb09e7c27060df73d4da34, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:49,508 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:55:49,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:49,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8570082a7f95a7fb8bb41d51fac7f3ee, NAME => 'testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.', STARTKEY => '', ENDKEY => '71'} 2023-02-10 07:55:49,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:49,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:49,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,521 INFO [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,522 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1 2023-02-10 07:55:49,522 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1 2023-02-10 07:55:49,522 INFO [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8570082a7f95a7fb8bb41d51fac7f3ee columnFamilyName cf1 2023-02-10 07:55:49,527 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-07e4d2212bc74b16956a4e9e57a671b6 2023-02-10 07:55:49,530 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-20391d5060c04593b0726c428467b759 2023-02-10 07:55:49,532 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-8e60fd11fb7544bfaf077ec61e61079e 2023-02-10 07:55:49,535 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-94bbe5706d004456aa6e219189c9d8aa 2023-02-10 07:55:49,537 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-b80c2b11aab748848824b0d498bfe026 2023-02-10 07:55:49,540 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-dd6282a2ab0d42cd975a2449212ff1a3 2023-02-10 07:55:49,542 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-ec9294852b24438aa60718b02430f84f 2023-02-10 07:55:49,542 INFO [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(310): Store=8570082a7f95a7fb8bb41d51fac7f3ee/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:49,542 INFO [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,543 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2 2023-02-10 07:55:49,543 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2 2023-02-10 07:55:49,544 INFO [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8570082a7f95a7fb8bb41d51fac7f3ee columnFamilyName cf2 2023-02-10 07:55:49,548 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-09406c0b35b3418da405f0c4c5745dcf 2023-02-10 07:55:49,550 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0da27882ec9747549f75504e28117a4f 2023-02-10 07:55:49,553 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0e92ec3fac1d4d689a8ca3b637c351b4 2023-02-10 07:55:49,555 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-518cf2552ce64ca59c8b63bcfde22c4f 2023-02-10 07:55:49,558 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-6cefa5af3b484804a21c3f2b1697d809 2023-02-10 07:55:49,560 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-949ab233837749f3a51fb15b1b8dcc45 2023-02-10 07:55:49,562 DEBUG [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-f8af2e2dd5c743fdb84946bab5705268 2023-02-10 07:55:49,562 INFO [StoreOpener-8570082a7f95a7fb8bb41d51fac7f3ee-1] regionserver.HStore(310): Store=8570082a7f95a7fb8bb41d51fac7f3ee/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:49,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:49,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:49,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8570082a7f95a7fb8bb41d51fac7f3ee; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71033128, jitterRate=0.05847609043121338}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:49,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8570082a7f95a7fb8bb41d51fac7f3ee: Running coprocessor pre-open hook at 1676015749520Writing region info on filesystem at 1676015749520Initializing all the Stores at 1676015749521 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015749521Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015749521Cleaning up temporary data from old regions at 1676015749564 (+43 ms)Cleaning up detritus from prior splits at 1676015749564Running coprocessor post-open hooks at 1676015749566 (+2 ms)Region opened successfully at 1676015749566 2023-02-10 07:55:49,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee., pid=133, masterSystemTime=1676015749517 2023-02-10 07:55:49,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:49,568 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:49,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:49,568 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:49,568 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-10 07:55:49,568 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-10 07:55:49,568 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1912): 8570082a7f95a7fb8bb41d51fac7f3ee/cf1 is initiating minor compaction (all files) 2023-02-10 07:55:49,568 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1912): 8570082a7f95a7fb8bb41d51fac7f3ee/cf2 is initiating minor compaction (all files) 2023-02-10 07:55:49,568 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8570082a7f95a7fb8bb41d51fac7f3ee/cf1 in testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:49,569 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8570082a7f95a7fb8bb41d51fac7f3ee/cf2 in testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:49,569 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-8e60fd11fb7544bfaf077ec61e61079e, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-20391d5060c04593b0726c428467b759, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-b80c2b11aab748848824b0d498bfe026, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-dd6282a2ab0d42cd975a2449212ff1a3, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-94bbe5706d004456aa6e219189c9d8aa, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-ec9294852b24438aa60718b02430f84f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-07e4d2212bc74b16956a4e9e57a671b6] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/.tmp, totalSize=34.9 K 2023-02-10 07:55:49,569 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0e92ec3fac1d4d689a8ca3b637c351b4, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-949ab233837749f3a51fb15b1b8dcc45, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0da27882ec9747549f75504e28117a4f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-518cf2552ce64ca59c8b63bcfde22c4f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-f8af2e2dd5c743fdb84946bab5705268, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-09406c0b35b3418da405f0c4c5745dcf, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-6cefa5af3b484804a21c3f2b1697d809] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/.tmp, totalSize=34.9 K 2023-02-10 07:55:49,569 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-8e60fd11fb7544bfaf077ec61e61079e, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015747768 2023-02-10 07:55:49,569 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0e92ec3fac1d4d689a8ca3b637c351b4, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015747768 2023-02-10 07:55:49,569 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-20391d5060c04593b0726c428467b759, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015747931 2023-02-10 07:55:49,569 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-949ab233837749f3a51fb15b1b8dcc45, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015747931 2023-02-10 07:55:49,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:49,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:49,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. 2023-02-10 07:55:49,570 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-b80c2b11aab748848824b0d498bfe026, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015748100 2023-02-10 07:55:49,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0da42b6033eb09e7c27060df73d4da34, NAME => 'testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.', STARTKEY => '71', ENDKEY => ''} 2023-02-10 07:55:49,570 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0da27882ec9747549f75504e28117a4f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015748100 2023-02-10 07:55:49,570 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=8570082a7f95a7fb8bb41d51fac7f3ee, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:49,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:49,570 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-dd6282a2ab0d42cd975a2449212ff1a3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015748279 2023-02-10 07:55:49,570 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749570"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015749570"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015749570"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015749570"}]},"ts":"1676015749570"} 2023-02-10 07:55:49,570 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-518cf2552ce64ca59c8b63bcfde22c4f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015748279 2023-02-10 07:55:49,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:49,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,571 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-94bbe5706d004456aa6e219189c9d8aa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015748448 2023-02-10 07:55:49,571 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-ec9294852b24438aa60718b02430f84f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015748619 2023-02-10 07:55:49,571 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-f8af2e2dd5c743fdb84946bab5705268, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015748448 2023-02-10 07:55:49,572 INFO [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,572 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-07e4d2212bc74b16956a4e9e57a671b6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015748804 2023-02-10 07:55:49,572 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-09406c0b35b3418da405f0c4c5745dcf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015748619 2023-02-10 07:55:49,573 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-6cefa5af3b484804a21c3f2b1697d809, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015748804 2023-02-10 07:55:49,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-10 07:55:49,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure 8570082a7f95a7fb8bb41d51fac7f3ee, server=jenkins-hbase4.apache.org,40867,1676015645736 in 206 msec 2023-02-10 07:55:49,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8570082a7f95a7fb8bb41d51fac7f3ee, ASSIGN in 363 msec 2023-02-10 07:55:49,577 DEBUG [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/cf1 2023-02-10 07:55:49,577 DEBUG [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/cf1 2023-02-10 07:55:49,578 INFO [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0da42b6033eb09e7c27060df73d4da34 columnFamilyName cf1 2023-02-10 07:55:49,578 INFO [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] regionserver.HStore(310): Store=0da42b6033eb09e7c27060df73d4da34/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:49,578 INFO [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,579 DEBUG [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/cf2 2023-02-10 07:55:49,580 DEBUG [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/cf2 2023-02-10 07:55:49,580 INFO [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0da42b6033eb09e7c27060df73d4da34 columnFamilyName cf2 2023-02-10 07:55:49,580 INFO [StoreOpener-0da42b6033eb09e7c27060df73d4da34-1] regionserver.HStore(310): Store=0da42b6033eb09e7c27060df73d4da34/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:49,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:49,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:49,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0da42b6033eb09e7c27060df73d4da34; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61105015, jitterRate=-0.08946432173252106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:49,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0da42b6033eb09e7c27060df73d4da34: Running coprocessor pre-open hook at 1676015749571Writing region info on filesystem at 1676015749571Initializing all the Stores at 1676015749572 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015749572Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015749572Cleaning up temporary data from old regions at 1676015749583 (+11 ms)Cleaning up detritus from prior splits at 1676015749583Running coprocessor post-open hooks at 1676015749586 (+3 ms)Region opened successfully at 1676015749586 2023-02-10 07:55:49,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34., pid=134, masterSystemTime=1676015749517 2023-02-10 07:55:49,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. 2023-02-10 07:55:49,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. 2023-02-10 07:55:49,589 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=0da42b6033eb09e7c27060df73d4da34, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:49,589 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749589"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015749589"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015749589"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015749589"}]},"ts":"1676015749589"} 2023-02-10 07:55:49,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-10 07:55:49,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure 0da42b6033eb09e7c27060df73d4da34, server=jenkins-hbase4.apache.org,40867,1676015645736 in 221 msec 2023-02-10 07:55:49,592 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8570082a7f95a7fb8bb41d51fac7f3ee#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:49,592 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] throttle.PressureAwareThroughputController(145): 8570082a7f95a7fb8bb41d51fac7f3ee#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:49,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-10 07:55:49,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0da42b6033eb09e7c27060df73d4da34, ASSIGN in 380 msec 2023-02-10 07:55:49,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=9b2eec727609515e00693d8091cd3cd5, daughterA=8570082a7f95a7fb8bb41d51fac7f3ee, daughterB=0da42b6033eb09e7c27060df73d4da34 in 786 msec 2023-02-10 07:55:49,614 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/.tmp/cf2/21da1a573b7f443fadbb48aa7579b2d3 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/21da1a573b7f443fadbb48aa7579b2d3 2023-02-10 07:55:49,616 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/.tmp/cf1/dfb3512631b04baca0a8e827165974bd as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/dfb3512631b04baca0a8e827165974bd 2023-02-10 07:55:49,619 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8570082a7f95a7fb8bb41d51fac7f3ee/cf2 of 8570082a7f95a7fb8bb41d51fac7f3ee into 21da1a573b7f443fadbb48aa7579b2d3(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:49,619 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8570082a7f95a7fb8bb41d51fac7f3ee: 2023-02-10 07:55:49,619 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee., storeName=8570082a7f95a7fb8bb41d51fac7f3ee/cf2, priority=9, startTime=1676015749568; duration=0sec 2023-02-10 07:55:49,619 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:49,620 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8570082a7f95a7fb8bb41d51fac7f3ee/cf1 of 8570082a7f95a7fb8bb41d51fac7f3ee into dfb3512631b04baca0a8e827165974bd(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:49,620 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8570082a7f95a7fb8bb41d51fac7f3ee: 2023-02-10 07:55:49,620 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee., storeName=8570082a7f95a7fb8bb41d51fac7f3ee/cf1, priority=9, startTime=1676015749567; duration=0sec 2023-02-10 07:55:49,621 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:49,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40867] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:47068 deadline: 1676015809812, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5. is not online on jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:49,922 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-10 07:55:49,925 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:49,926 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-10 07:55:49,927 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-10 07:55:49,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:49,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-10 07:55:49,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015749930"}]},"ts":"1676015749930"} 2023-02-10 07:55:49,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-10 07:55:49,934 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-10 07:55:49,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8570082a7f95a7fb8bb41d51fac7f3ee, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0da42b6033eb09e7c27060df73d4da34, UNASSIGN}] 2023-02-10 07:55:49,935 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0da42b6033eb09e7c27060df73d4da34, UNASSIGN 2023-02-10 07:55:49,935 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=0da42b6033eb09e7c27060df73d4da34, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:49,935 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749935"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015749935"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015749935"}]},"ts":"1676015749935"} 2023-02-10 07:55:49,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 0da42b6033eb09e7c27060df73d4da34, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:49,938 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8570082a7f95a7fb8bb41d51fac7f3ee, UNASSIGN 2023-02-10 07:55:49,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=8570082a7f95a7fb8bb41d51fac7f3ee, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:49,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015749939"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015749939"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015749939"}]},"ts":"1676015749939"} 2023-02-10 07:55:49,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 8570082a7f95a7fb8bb41d51fac7f3ee, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:50,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-10 07:55:50,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:50,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8570082a7f95a7fb8bb41d51fac7f3ee, disabling compactions & flushes 2023-02-10 07:55:50,088 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:50,088 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:50,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. after waiting 0 ms 2023-02-10 07:55:50,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:50,089 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-8e60fd11fb7544bfaf077ec61e61079e, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-20391d5060c04593b0726c428467b759, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-b80c2b11aab748848824b0d498bfe026, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-dd6282a2ab0d42cd975a2449212ff1a3, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-94bbe5706d004456aa6e219189c9d8aa, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-ec9294852b24438aa60718b02430f84f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-07e4d2212bc74b16956a4e9e57a671b6] to archive 2023-02-10 07:55:50,090 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:50,092 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-8e60fd11fb7544bfaf077ec61e61079e to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-8e60fd11fb7544bfaf077ec61e61079e 2023-02-10 07:55:50,093 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-20391d5060c04593b0726c428467b759 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-20391d5060c04593b0726c428467b759 2023-02-10 07:55:50,094 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-b80c2b11aab748848824b0d498bfe026 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-b80c2b11aab748848824b0d498bfe026 2023-02-10 07:55:50,095 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-dd6282a2ab0d42cd975a2449212ff1a3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-dd6282a2ab0d42cd975a2449212ff1a3 2023-02-10 07:55:50,095 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-94bbe5706d004456aa6e219189c9d8aa to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-94bbe5706d004456aa6e219189c9d8aa 2023-02-10 07:55:50,096 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-ec9294852b24438aa60718b02430f84f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-ec9294852b24438aa60718b02430f84f 2023-02-10 07:55:50,097 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-07e4d2212bc74b16956a4e9e57a671b6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-07e4d2212bc74b16956a4e9e57a671b6 2023-02-10 07:55:50,098 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0e92ec3fac1d4d689a8ca3b637c351b4, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-949ab233837749f3a51fb15b1b8dcc45, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0da27882ec9747549f75504e28117a4f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-518cf2552ce64ca59c8b63bcfde22c4f, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-f8af2e2dd5c743fdb84946bab5705268, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-09406c0b35b3418da405f0c4c5745dcf, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-6cefa5af3b484804a21c3f2b1697d809] to archive 2023-02-10 07:55:50,099 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:50,100 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0e92ec3fac1d4d689a8ca3b637c351b4 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0e92ec3fac1d4d689a8ca3b637c351b4 2023-02-10 07:55:50,101 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-949ab233837749f3a51fb15b1b8dcc45 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-949ab233837749f3a51fb15b1b8dcc45 2023-02-10 07:55:50,102 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0da27882ec9747549f75504e28117a4f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-0da27882ec9747549f75504e28117a4f 2023-02-10 07:55:50,103 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-518cf2552ce64ca59c8b63bcfde22c4f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-518cf2552ce64ca59c8b63bcfde22c4f 2023-02-10 07:55:50,104 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-f8af2e2dd5c743fdb84946bab5705268 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-f8af2e2dd5c743fdb84946bab5705268 2023-02-10 07:55:50,105 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-09406c0b35b3418da405f0c4c5745dcf to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-09406c0b35b3418da405f0c4c5745dcf 2023-02-10 07:55:50,106 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-6cefa5af3b484804a21c3f2b1697d809 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/testSplitTableRegionEmptyDaughter=9b2eec727609515e00693d8091cd3cd5-6cefa5af3b484804a21c3f2b1697d809 2023-02-10 07:55:50,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:55:50,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:50,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee. 2023-02-10 07:55:50,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8570082a7f95a7fb8bb41d51fac7f3ee: Waiting for close lock at 1676015750088Running coprocessor pre-close hooks at 1676015750088Disabling compacts and flushes for region at 1676015750088Disabling writes for close at 1676015750089 (+1 ms)Writing region close event to WAL at 1676015750106 (+17 ms)Running coprocessor post-close hooks at 1676015750112 (+6 ms)Closed at 1676015750113 (+1 ms) 2023-02-10 07:55:50,114 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:50,114 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:50,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0da42b6033eb09e7c27060df73d4da34, disabling compactions & flushes 2023-02-10 07:55:50,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. 2023-02-10 07:55:50,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. 2023-02-10 07:55:50,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. after waiting 0 ms 2023-02-10 07:55:50,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. 2023-02-10 07:55:50,116 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=8570082a7f95a7fb8bb41d51fac7f3ee, regionState=CLOSED 2023-02-10 07:55:50,116 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015750116"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015750116"}]},"ts":"1676015750116"} 2023-02-10 07:55:50,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-10 07:55:50,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-10 07:55:50,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 8570082a7f95a7fb8bb41d51fac7f3ee, server=jenkins-hbase4.apache.org,40867,1676015645736 in 175 msec 2023-02-10 07:55:50,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:50,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34. 2023-02-10 07:55:50,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0da42b6033eb09e7c27060df73d4da34: Waiting for close lock at 1676015750114Running coprocessor pre-close hooks at 1676015750114Disabling compacts and flushes for region at 1676015750115 (+1 ms)Disabling writes for close at 1676015750116 (+1 ms)Writing region close event to WAL at 1676015750116Running coprocessor post-close hooks at 1676015750119 (+3 ms)Closed at 1676015750119 2023-02-10 07:55:50,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8570082a7f95a7fb8bb41d51fac7f3ee, UNASSIGN in 185 msec 2023-02-10 07:55:50,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:50,121 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=0da42b6033eb09e7c27060df73d4da34, regionState=CLOSED 2023-02-10 07:55:50,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676015750121"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015750121"}]},"ts":"1676015750121"} 2023-02-10 07:55:50,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-10 07:55:50,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 0da42b6033eb09e7c27060df73d4da34, server=jenkins-hbase4.apache.org,40867,1676015645736 in 186 msec 2023-02-10 07:55:50,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-10 07:55:50,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=0da42b6033eb09e7c27060df73d4da34, UNASSIGN in 189 msec 2023-02-10 07:55:50,125 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015750125"}]},"ts":"1676015750125"} 2023-02-10 07:55:50,126 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-10 07:55:50,127 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-10 07:55:50,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 201 msec 2023-02-10 07:55:50,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-10 07:55:50,233 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-10 07:55:50,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-10 07:55:50,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:50,235 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:50,236 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:50,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-10 07:55:50,240 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:50,240 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:50,240 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:50,242 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/recovered.edits] 2023-02-10 07:55:50,242 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/recovered.edits] 2023-02-10 07:55:50,242 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/recovered.edits] 2023-02-10 07:55:50,245 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/dfb3512631b04baca0a8e827165974bd to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf1/dfb3512631b04baca0a8e827165974bd 2023-02-10 07:55:50,245 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/07e4d2212bc74b16956a4e9e57a671b6 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/07e4d2212bc74b16956a4e9e57a671b6 2023-02-10 07:55:50,246 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/20391d5060c04593b0726c428467b759 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/20391d5060c04593b0726c428467b759 2023-02-10 07:55:50,247 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/recovered.edits/88.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34/recovered.edits/88.seqid 2023-02-10 07:55:50,247 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/21da1a573b7f443fadbb48aa7579b2d3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/cf2/21da1a573b7f443fadbb48aa7579b2d3 2023-02-10 07:55:50,247 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/8e60fd11fb7544bfaf077ec61e61079e to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/8e60fd11fb7544bfaf077ec61e61079e 2023-02-10 07:55:50,248 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/0da42b6033eb09e7c27060df73d4da34 2023-02-10 07:55:50,249 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/94bbe5706d004456aa6e219189c9d8aa to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/94bbe5706d004456aa6e219189c9d8aa 2023-02-10 07:55:50,250 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee/recovered.edits/90.seqid 2023-02-10 07:55:50,250 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/b80c2b11aab748848824b0d498bfe026 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/b80c2b11aab748848824b0d498bfe026 2023-02-10 07:55:50,250 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/8570082a7f95a7fb8bb41d51fac7f3ee 2023-02-10 07:55:50,251 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/dd6282a2ab0d42cd975a2449212ff1a3 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/dd6282a2ab0d42cd975a2449212ff1a3 2023-02-10 07:55:50,252 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/ec9294852b24438aa60718b02430f84f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf1/ec9294852b24438aa60718b02430f84f 2023-02-10 07:55:50,254 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/09406c0b35b3418da405f0c4c5745dcf to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/09406c0b35b3418da405f0c4c5745dcf 2023-02-10 07:55:50,255 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0da27882ec9747549f75504e28117a4f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0da27882ec9747549f75504e28117a4f 2023-02-10 07:55:50,256 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0e92ec3fac1d4d689a8ca3b637c351b4 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/0e92ec3fac1d4d689a8ca3b637c351b4 2023-02-10 07:55:50,257 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/518cf2552ce64ca59c8b63bcfde22c4f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/518cf2552ce64ca59c8b63bcfde22c4f 2023-02-10 07:55:50,258 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/6cefa5af3b484804a21c3f2b1697d809 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/6cefa5af3b484804a21c3f2b1697d809 2023-02-10 07:55:50,259 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/949ab233837749f3a51fb15b1b8dcc45 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/949ab233837749f3a51fb15b1b8dcc45 2023-02-10 07:55:50,260 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/f8af2e2dd5c743fdb84946bab5705268 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/cf2/f8af2e2dd5c743fdb84946bab5705268 2023-02-10 07:55:50,262 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/recovered.edits/85.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5/recovered.edits/85.seqid 2023-02-10 07:55:50,262 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegionEmptyDaughter/9b2eec727609515e00693d8091cd3cd5 2023-02-10 07:55:50,262 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-10 07:55:50,264 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:50,266 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-10 07:55:50,267 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-10 07:55:50,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:50,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-10 07:55:50,269 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015750268"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:50,269 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015750268"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:50,269 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015750268"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:50,270 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-10 07:55:50,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9b2eec727609515e00693d8091cd3cd5, NAME => 'testSplitTableRegionEmptyDaughter,,1676015747265.9b2eec727609515e00693d8091cd3cd5.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 8570082a7f95a7fb8bb41d51fac7f3ee, NAME => 'testSplitTableRegionEmptyDaughter,,1676015748810.8570082a7f95a7fb8bb41d51fac7f3ee.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 0da42b6033eb09e7c27060df73d4da34, NAME => 'testSplitTableRegionEmptyDaughter,71,1676015748810.0da42b6033eb09e7c27060df73d4da34.', STARTKEY => '71', ENDKEY => ''}] 2023-02-10 07:55:50,270 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-10 07:55:50,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015750270"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:50,271 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-10 07:55:50,273 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-10 07:55:50,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 40 msec 2023-02-10 07:55:50,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-10 07:55:50,337 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-10 07:55:50,350 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377 (was 378), OpenFileDescriptor=827 (was 829), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 87) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3142 (was 3154) 2023-02-10 07:55:50,364 INFO [Listener at localhost/43795] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377, OpenFileDescriptor=827, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=168, AvailableMemoryMB=3141 2023-02-10 07:55:50,364 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:50,368 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-10 07:55:50,368 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-10 07:55:50,369 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-10 07:55:50,371 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-10 07:55:50,371 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-10 07:55:50,372 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,373 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2 empty. 2023-02-10 07:55:50,373 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,373 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-10 07:55:50,382 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-10 07:55:50,383 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7a0f0f141e6bd5bed887272a2c1462d2, NAME => 'testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp 2023-02-10 07:55:50,389 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:50,389 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 7a0f0f141e6bd5bed887272a2c1462d2, disabling compactions & flushes 2023-02-10 07:55:50,389 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,389 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,389 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. after waiting 0 ms 2023-02-10 07:55:50,389 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,389 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,389 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 7a0f0f141e6bd5bed887272a2c1462d2: Waiting for close lock at 1676015750389Disabling compacts and flushes for region at 1676015750389Disabling writes for close at 1676015750389Writing region close event to WAL at 1676015750389Closed at 1676015750389 2023-02-10 07:55:50,391 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-10 07:55:50,391 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015750391"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015750391"}]},"ts":"1676015750391"} 2023-02-10 07:55:50,392 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-10 07:55:50,393 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-10 07:55:50,393 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015750393"}]},"ts":"1676015750393"} 2023-02-10 07:55:50,394 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-10 07:55:50,397 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-10 07:55:50,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-10 07:55:50,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-10 07:55:50,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-10 07:55:50,398 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-10 07:55:50,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a0f0f141e6bd5bed887272a2c1462d2, ASSIGN}] 2023-02-10 07:55:50,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a0f0f141e6bd5bed887272a2c1462d2, ASSIGN 2023-02-10 07:55:50,399 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=7a0f0f141e6bd5bed887272a2c1462d2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:55:50,549 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-10 07:55:50,551 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=7a0f0f141e6bd5bed887272a2c1462d2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015750551"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015750551"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015750551"}]},"ts":"1676015750551"} 2023-02-10 07:55:50,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 7a0f0f141e6bd5bed887272a2c1462d2, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:50,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7a0f0f141e6bd5bed887272a2c1462d2, NAME => 'testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.', STARTKEY => '', ENDKEY => ''} 2023-02-10 07:55:50,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:50,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:50,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,708 INFO [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,709 DEBUG [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1 2023-02-10 07:55:50,709 DEBUG [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1 2023-02-10 07:55:50,709 INFO [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7a0f0f141e6bd5bed887272a2c1462d2 columnFamilyName cf1 2023-02-10 07:55:50,710 INFO [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] regionserver.HStore(310): Store=7a0f0f141e6bd5bed887272a2c1462d2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:50,710 INFO [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,711 DEBUG [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2 2023-02-10 07:55:50,711 DEBUG [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2 2023-02-10 07:55:50,711 INFO [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7a0f0f141e6bd5bed887272a2c1462d2 columnFamilyName cf2 2023-02-10 07:55:50,712 INFO [StoreOpener-7a0f0f141e6bd5bed887272a2c1462d2-1] regionserver.HStore(310): Store=7a0f0f141e6bd5bed887272a2c1462d2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:50,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:50,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:50,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-10 07:55:50,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7a0f0f141e6bd5bed887272a2c1462d2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63533586, jitterRate=-0.05327579379081726}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:50,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7a0f0f141e6bd5bed887272a2c1462d2: Running coprocessor pre-open hook at 1676015750707Writing region info on filesystem at 1676015750707Initializing all the Stores at 1676015750707Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015750707Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015750708 (+1 ms)Cleaning up temporary data from old regions at 1676015750713 (+5 ms)Cleaning up detritus from prior splits at 1676015750713Running coprocessor post-open hooks at 1676015750716 (+3 ms)Region opened successfully at 1676015750716 2023-02-10 07:55:50,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2., pid=143, masterSystemTime=1676015750703 2023-02-10 07:55:50,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,718 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=7a0f0f141e6bd5bed887272a2c1462d2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,719 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015750718"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015750718"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015750718"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015750718"}]},"ts":"1676015750718"} 2023-02-10 07:55:50,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-10 07:55:50,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 7a0f0f141e6bd5bed887272a2c1462d2, server=jenkins-hbase4.apache.org,40867,1676015645736 in 168 msec 2023-02-10 07:55:50,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-10 07:55:50,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7a0f0f141e6bd5bed887272a2c1462d2, ASSIGN in 323 msec 2023-02-10 07:55:50,723 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-10 07:55:50,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015750723"}]},"ts":"1676015750723"} 2023-02-10 07:55:50,724 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-10 07:55:50,725 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-10 07:55:50,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 357 msec 2023-02-10 07:55:50,876 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:50,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-10 07:55:50,878 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-10 07:55:50,878 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:50,878 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-10 07:55:50,878 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-10 07:55:50,878 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,878 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,879 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:50,879 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:50,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:50,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:50,879 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,879 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:50,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:50,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,880 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,880 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:50,881 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:50,882 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:50,881 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,881 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:50,882 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. started... 2023-02-10 07:55:50,883 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a0f0f141e6bd5bed887272a2c1462d2 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-10 07:55:50,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,890 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:50,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,890 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:50,892 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/1ea0116c67d542dc965f226537878d9a 2023-02-10 07:55:50,904 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/9bd55aa75b4b46798b9c502111fc6a8d 2023-02-10 07:55:50,907 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/1ea0116c67d542dc965f226537878d9a as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a 2023-02-10 07:55:50,910 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:50,911 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/9bd55aa75b4b46798b9c502111fc6a8d as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d 2023-02-10 07:55:50,914 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:50,915 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 32ms, sequenceid=6, compaction requested=false 2023-02-10 07:55:50,915 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-10 07:55:50,915 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a0f0f141e6bd5bed887272a2c1462d2: 2023-02-10 07:55:50,915 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:50,915 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:50,915 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:50,915 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,915 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:50,916 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:50,917 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,917 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:50,917 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,917 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:50,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:50,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:50,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:50,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-10 07:55:50,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30030e60[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:50,919 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-10 07:55:50,919 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:50,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:50,920 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,920 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:50,920 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:50,921 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,921 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:50,922 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:50,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:50,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:50,922 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:50,925 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:50,925 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:50,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:50,926 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:50,925 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:50,926 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:50,926 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:50,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:50,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:50,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:50,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:50,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:50,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-10 07:55:50,928 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-10 07:55:50,928 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:50,928 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:50,928 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-10 07:55:50,928 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:50,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:50,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:50,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:50,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,930 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:50,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:50,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:50,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:50,931 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:50,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:50,931 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,933 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:50,933 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:50,933 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:50,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:50,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:50,936 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-10 07:55:50,936 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:50,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:50,936 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:50,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:50,935 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:50,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:50,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:50,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:50,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:50,936 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:50,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:50,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:50,937 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:50,937 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,036 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:51,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:51,047 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:51,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:51,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-10 07:55:51,048 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-10 07:55:51,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-10 07:55:51,048 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-10 07:55:51,049 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,049 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,051 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,051 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,051 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,051 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,051 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,051 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:51,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,052 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:51,053 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,053 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. started... 2023-02-10 07:55:51,053 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a0f0f141e6bd5bed887272a2c1462d2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:51,054 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,066 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/9828c324f19d45d983ffec661c06ddf9 2023-02-10 07:55:51,078 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/c81ee2be012143c69382bfd5ad6a3779 2023-02-10 07:55:51,082 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/9828c324f19d45d983ffec661c06ddf9 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9 2023-02-10 07:55:51,085 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:51,086 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/c81ee2be012143c69382bfd5ad6a3779 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779 2023-02-10 07:55:51,088 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779, entries=10, sequenceid=19, filesize=5.0 K 2023-02-10 07:55:51,089 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 36ms, sequenceid=19, compaction requested=false 2023-02-10 07:55:51,089 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a0f0f141e6bd5bed887272a2c1462d2: 2023-02-10 07:55:51,089 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,089 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:51,089 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:51,089 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,089 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,090 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,091 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,091 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,091 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,091 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-10 07:55:51,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c10f8bd[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:51,093 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-10 07:55:51,093 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,095 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,095 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:51,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,096 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:51,096 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,096 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,098 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,098 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,098 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,098 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,098 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,099 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-10 07:55:51,101 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-10 07:55:51,101 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:51,101 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:51,101 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-10 07:55:51,101 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:51,103 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,103 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,103 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,103 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,104 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,104 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,105 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,106 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,106 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,109 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,109 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,109 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-10 07:55:51,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,110 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,110 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:51,111 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:51,211 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:51,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:51,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:51,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:51,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-10 07:55:51,223 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-10 07:55:51,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-10 07:55:51,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-10 07:55:51,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,223 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,225 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,225 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,225 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,225 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,226 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,226 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,226 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,226 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,227 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,226 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:51,227 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:51,227 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,227 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. started... 2023-02-10 07:55:51,227 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a0f0f141e6bd5bed887272a2c1462d2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:51,228 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,232 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,232 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,233 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,233 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,233 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,238 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/a9ba015ce5c6498a94c8ad95e5446524 2023-02-10 07:55:51,248 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/cccac44aaa934934bfd5efb2a8408f48 2023-02-10 07:55:51,251 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/a9ba015ce5c6498a94c8ad95e5446524 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524 2023-02-10 07:55:51,254 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:51,254 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/cccac44aaa934934bfd5efb2a8408f48 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48 2023-02-10 07:55:51,257 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48, entries=10, sequenceid=32, filesize=5.0 K 2023-02-10 07:55:51,258 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 31ms, sequenceid=32, compaction requested=true 2023-02-10 07:55:51,258 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a0f0f141e6bd5bed887272a2c1462d2: 2023-02-10 07:55:51,258 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,260 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,260 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-10 07:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6de54fc4[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:51,262 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-10 07:55:51,262 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:51,263 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,263 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,263 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:51,266 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:51,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-10 07:55:51,270 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-10 07:55:51,270 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:51,270 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:51,270 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-10 07:55:51,270 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:51,271 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,271 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,271 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,271 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,271 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,272 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,272 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,272 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,273 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,273 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,276 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,276 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,277 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,280 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,280 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-10 07:55:51,281 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:51,280 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:51,281 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:51,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,381 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:51,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:51,397 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:51,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:51,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-10 07:55:51,399 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-10 07:55:51,399 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,399 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-10 07:55:51,399 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-10 07:55:51,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,400 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,402 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,402 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,402 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,402 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,403 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,403 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:51,403 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,403 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,406 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,407 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,407 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,407 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,407 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:51,407 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:51,407 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,408 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. started... 2023-02-10 07:55:51,408 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a0f0f141e6bd5bed887272a2c1462d2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:51,409 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,409 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,409 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,426 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/9f6533583ec34b6fbfb634fedce7083e 2023-02-10 07:55:51,442 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd 2023-02-10 07:55:51,446 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/9f6533583ec34b6fbfb634fedce7083e as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e 2023-02-10 07:55:51,449 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:51,449 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd 2023-02-10 07:55:51,453 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-10 07:55:51,453 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 45ms, sequenceid=45, compaction requested=true 2023-02-10 07:55:51,454 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a0f0f141e6bd5bed887272a2c1462d2: 2023-02-10 07:55:51,454 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,454 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:51,454 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:51,454 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,454 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,454 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,455 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,455 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,456 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,456 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-10 07:55:51,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1155cd00[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:51,458 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-10 07:55:51,458 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,463 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,463 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,463 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,465 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,465 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:51,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-10 07:55:51,469 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-10 07:55:51,469 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:51,469 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:51,469 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-10 07:55:51,469 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:51,471 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,471 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,471 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,471 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,472 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,472 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,473 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,474 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,475 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,475 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,481 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-10 07:55:51,482 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:51,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,481 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,482 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,482 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:51,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,482 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,482 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:51,482 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,483 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,483 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,483 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,483 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,483 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,483 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,583 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:51,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:51,594 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:51,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:51,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-10 07:55:51,595 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-10 07:55:51,595 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,595 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-10 07:55:51,595 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-10 07:55:51,596 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,596 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,597 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,597 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,597 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,597 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,597 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,597 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:51,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,598 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:51,599 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,599 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. started... 2023-02-10 07:55:51,599 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a0f0f141e6bd5bed887272a2c1462d2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:51,600 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,600 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,601 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,601 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,601 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,601 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,607 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/1e9553fb84f34311ad19dde00b899c09 2023-02-10 07:55:51,617 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/c64417e9689842f8a0e4bb3e2df8ba3f 2023-02-10 07:55:51,621 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/1e9553fb84f34311ad19dde00b899c09 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09 2023-02-10 07:55:51,624 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:51,624 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/c64417e9689842f8a0e4bb3e2df8ba3f as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f 2023-02-10 07:55:51,627 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-10 07:55:51,628 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 29ms, sequenceid=58, compaction requested=true 2023-02-10 07:55:51,628 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a0f0f141e6bd5bed887272a2c1462d2: 2023-02-10 07:55:51,628 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,628 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:51,629 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:51,629 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,629 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,629 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,631 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,631 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,632 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,632 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-10 07:55:51,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15ef8d35[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:51,633 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-10 07:55:51,633 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,634 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,634 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,635 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,635 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:51,636 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,636 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,636 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,637 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,637 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:51,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-10 07:55:51,640 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-10 07:55:51,640 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:51,640 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:51,640 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-10 07:55:51,640 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:51,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,643 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,642 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,643 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,643 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,644 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,645 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,645 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,647 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,647 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,647 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,647 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,647 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-10 07:55:51,647 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:51,647 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:51,648 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:51,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,647 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,647 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,652 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,652 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,652 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,652 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,653 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-02-10 07:55:51,748 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:51,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:51,760 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-10 07:55:51,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-10 07:55:51,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-10 07:55:51,761 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-10 07:55:51,761 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,761 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-10 07:55:51,761 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-10 07:55:51,762 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,762 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,763 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,763 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,763 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,763 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,764 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,764 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-10 07:55:51,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,765 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,765 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-10 07:55:51,765 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,765 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,765 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. started... 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,765 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-10 07:55:51,765 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43281,1676015645662' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-10 07:55:51,766 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a0f0f141e6bd5bed887272a2c1462d2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-10 07:55:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-10 07:55:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-10 07:55:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-10 07:55:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,766 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43211,1676015645706' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,767 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,768 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,768 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,768 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,769 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,769 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,777 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/d9ce9c92b3894f07811b5749db167d70 2023-02-10 07:55:51,787 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/aa2f6c2ec0374972b13245df645cd07a 2023-02-10 07:55:51,791 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/d9ce9c92b3894f07811b5749db167d70 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70 2023-02-10 07:55:51,794 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:51,795 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/aa2f6c2ec0374972b13245df645cd07a as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a 2023-02-10 07:55:51,798 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-10 07:55:51,798 INFO [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 33ms, sequenceid=71, compaction requested=true 2023-02-10 07:55:51,798 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a0f0f141e6bd5bed887272a2c1462d2: 2023-02-10 07:55:51,798 DEBUG [rs(jenkins-hbase4.apache.org,40867,1676015645736)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:51,798 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-10 07:55:51,798 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-10 07:55:51,798 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,798 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-10 07:55:51,798 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-10 07:55:51,800 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,800 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,800 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,800 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-10 07:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,40867,1676015645736' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-10 07:55:51,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a6d6f45[Count = 0] remaining members to acquire global barrier 2023-02-10 07:55:51,802 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-10 07:55:51,802 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,40867,1676015645736' in zk 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-10 07:55:51,804 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43281,1676015645662' in zk 2023-02-10 07:55:51,803 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,804 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-10 07:55:51,803 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43211,1676015645706' in zk 2023-02-10 07:55:51,810 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,810 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,810 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,810 DEBUG [member: 'jenkins-hbase4.apache.org,40867,1676015645736' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,811 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,811 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,811 DEBUG [member: 'jenkins-hbase4.apache.org,43281,1676015645662' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,811 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-10 07:55:51,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,811 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,811 DEBUG [member: 'jenkins-hbase4.apache.org,43211,1676015645706' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-10 07:55:51,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,40867,1676015645736': 2023-02-10 07:55:51,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,40867,1676015645736' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-10 07:55:51,815 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-10 07:55:51,815 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-10 07:55:51,815 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-10 07:55:51,815 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-10 07:55:51,815 INFO [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-10 07:55:51,817 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-10 07:55:51,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-10 07:55:51,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,817 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-10 07:55:51,818 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,818 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-10 07:55:51,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,819 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-10 07:55:51,820 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-10 07:55:51,821 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,821 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-10 07:55:51,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-10 07:55:51,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-10 07:55:51,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-10 07:55:51,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-10 07:55:51,826 DEBUG [(jenkins-hbase4.apache.org,37109,1676015644499)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-10 07:55:51,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,827 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-10 07:55:51,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,826 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:51,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-10 07:55:51,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-10 07:55:51,827 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,827 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-10 07:55:51,827 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:51,827 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,827 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-10 07:55:51,827 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-10 07:55:51,927 DEBUG [Listener at localhost/43795] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-10 07:55:51,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-10 07:55:51,934 DEBUG [Listener at localhost/43795] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,935 DEBUG [Listener at localhost/43795] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a0f0f141e6bd5bed887272a2c1462d2, daughterA=c4d2ecf65a7239afbea796e4353ab457, daughterB=33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:51,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a0f0f141e6bd5bed887272a2c1462d2, daughterA=c4d2ecf65a7239afbea796e4353ab457, daughterB=33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:51,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a0f0f141e6bd5bed887272a2c1462d2, daughterA=c4d2ecf65a7239afbea796e4353ab457, daughterB=33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:51,936 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a0f0f141e6bd5bed887272a2c1462d2, daughterA=c4d2ecf65a7239afbea796e4353ab457, daughterB=33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:51,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a0f0f141e6bd5bed887272a2c1462d2, UNASSIGN}] 2023-02-10 07:55:51,938 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a0f0f141e6bd5bed887272a2c1462d2, UNASSIGN 2023-02-10 07:55:51,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=7a0f0f141e6bd5bed887272a2c1462d2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:51,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015751939"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015751939"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015751939"}]},"ts":"1676015751939"} 2023-02-10 07:55:51,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 7a0f0f141e6bd5bed887272a2c1462d2, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:52,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7a0f0f141e6bd5bed887272a2c1462d2, disabling compactions & flushes 2023-02-10 07:55:52,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:52,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:52,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. after waiting 0 ms 2023-02-10 07:55:52,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:52,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 7a0f0f141e6bd5bed887272a2c1462d2 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-10 07:55:52,104 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/5e2085a2592e4c318442ef500c1116cd 2023-02-10 07:55:52,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/7aa5da8155174f658fe3a44a9c44b608 2023-02-10 07:55:52,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf1/5e2085a2592e4c318442ef500c1116cd as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd 2023-02-10 07:55:52,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:52,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/.tmp/cf2/7aa5da8155174f658fe3a44a9c44b608 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608 2023-02-10 07:55:52,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608, entries=8, sequenceid=82, filesize=5.0 K 2023-02-10 07:55:52,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 32ms, sequenceid=82, compaction requested=true 2023-02-10 07:55:52,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-10 07:55:52,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:52,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. 2023-02-10 07:55:52,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7a0f0f141e6bd5bed887272a2c1462d2: Waiting for close lock at 1676015752091Running coprocessor pre-close hooks at 1676015752091Disabling compacts and flushes for region at 1676015752091Disabling writes for close at 1676015752092 (+1 ms)Obtaining lock to block concurrent updates at 1676015752092Preparing flush snapshotting stores in 7a0f0f141e6bd5bed887272a2c1462d2 at 1676015752092Finished memstore snapshotting testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676015752092Flushing stores of testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. at 1676015752092Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf1: creating writer at 1676015752093 (+1 ms)Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf1: appending metadata at 1676015752095 (+2 ms)Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf1: closing flushed file at 1676015752095Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf2: creating writer at 1676015752107 (+12 ms)Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf2: appending metadata at 1676015752108 (+1 ms)Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf2: closing flushed file at 1676015752108Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf1: reopening flushed file at 1676015752117 (+9 ms)Flushing 7a0f0f141e6bd5bed887272a2c1462d2/cf2: reopening flushed file at 1676015752121 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7a0f0f141e6bd5bed887272a2c1462d2 in 32ms, sequenceid=82, compaction requested=true at 1676015752124 (+3 ms)Writing region close event to WAL at 1676015752126 (+2 ms)Running coprocessor post-close hooks at 1676015752128 (+2 ms)Closed at 1676015752128 2023-02-10 07:55:52,130 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=7a0f0f141e6bd5bed887272a2c1462d2, regionState=CLOSED 2023-02-10 07:55:52,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015752130"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015752130"}]},"ts":"1676015752130"} 2023-02-10 07:55:52,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-10 07:55:52,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 7a0f0f141e6bd5bed887272a2c1462d2, server=jenkins-hbase4.apache.org,40867,1676015645736 in 191 msec 2023-02-10 07:55:52,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-10 07:55:52,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7a0f0f141e6bd5bed887272a2c1462d2, UNASSIGN in 194 msec 2023-02-10 07:55:52,141 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=7a0f0f141e6bd5bed887272a2c1462d2, threads=14 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,142 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,143 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,143 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,143 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,143 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,143 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,143 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,291 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,298 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,338 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,341 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,347 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,348 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,352 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,352 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,362 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,368 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,372 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,380 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,384 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,387 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9 for region: 7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:52,387 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 7a0f0f141e6bd5bed887272a2c1462d2 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-10 07:55:52,424 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:55:52,426 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-10 07:55:52,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676015752427"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676015752427"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676015752427"}]},"ts":"1676015752427"} 2023-02-10 07:55:52,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015752427"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015752427"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015752427"}]},"ts":"1676015752427"} 2023-02-10 07:55:52,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015752427"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015752427"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015752427"}]},"ts":"1676015752427"} 2023-02-10 07:55:52,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c4d2ecf65a7239afbea796e4353ab457, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=33f54e6bb3ee1980dafaaea7a567c33c, ASSIGN}] 2023-02-10 07:55:52,430 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=33f54e6bb3ee1980dafaaea7a567c33c, ASSIGN 2023-02-10 07:55:52,430 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=33f54e6bb3ee1980dafaaea7a567c33c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:55:52,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c4d2ecf65a7239afbea796e4353ab457, ASSIGN 2023-02-10 07:55:52,431 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=c4d2ecf65a7239afbea796e4353ab457, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,40867,1676015645736; forceNewPlan=false, retain=false 2023-02-10 07:55:52,581 INFO [jenkins-hbase4:37109] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-10 07:55:52,582 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c4d2ecf65a7239afbea796e4353ab457, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:52,582 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015752582"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015752582"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015752582"}]},"ts":"1676015752582"} 2023-02-10 07:55:52,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure c4d2ecf65a7239afbea796e4353ab457, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:52,586 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=33f54e6bb3ee1980dafaaea7a567c33c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:52,586 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015752586"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015752586"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015752586"}]},"ts":"1676015752586"} 2023-02-10 07:55:52,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 33f54e6bb3ee1980dafaaea7a567c33c, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:52,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:52,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 33f54e6bb3ee1980dafaaea7a567c33c, NAME => 'testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.', STARTKEY => '41', ENDKEY => ''} 2023-02-10 07:55:52,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:52,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:52,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,739 INFO [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,740 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1 2023-02-10 07:55:52,740 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1 2023-02-10 07:55:52,740 INFO [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33f54e6bb3ee1980dafaaea7a567c33c columnFamilyName cf1 2023-02-10 07:55:52,751 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09-top 2023-02-10 07:55:52,753 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a-top 2023-02-10 07:55:52,756 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd-top 2023-02-10 07:55:52,759 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9-top 2023-02-10 07:55:52,762 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e-top 2023-02-10 07:55:52,765 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524-top 2023-02-10 07:55:52,767 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70-top 2023-02-10 07:55:52,767 INFO [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(310): Store=33f54e6bb3ee1980dafaaea7a567c33c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:52,768 INFO [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,768 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2 2023-02-10 07:55:52,768 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2 2023-02-10 07:55:52,769 INFO [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33f54e6bb3ee1980dafaaea7a567c33c columnFamilyName cf2 2023-02-10 07:55:52,779 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd-top 2023-02-10 07:55:52,781 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608-top 2023-02-10 07:55:52,784 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d-top 2023-02-10 07:55:52,787 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a-top 2023-02-10 07:55:52,789 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f-top 2023-02-10 07:55:52,793 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779-top 2023-02-10 07:55:52,795 DEBUG [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48-top 2023-02-10 07:55:52,796 INFO [StoreOpener-33f54e6bb3ee1980dafaaea7a567c33c-1] regionserver.HStore(310): Store=33f54e6bb3ee1980dafaaea7a567c33c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:52,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:52,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:52,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 33f54e6bb3ee1980dafaaea7a567c33c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66978546, jitterRate=-0.0019418895244598389}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:52,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 33f54e6bb3ee1980dafaaea7a567c33c: Running coprocessor pre-open hook at 1676015752738Writing region info on filesystem at 1676015752738Initializing all the Stores at 1676015752739 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015752739Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015752739Cleaning up temporary data from old regions at 1676015752798 (+59 ms)Cleaning up detritus from prior splits at 1676015752798Running coprocessor post-open hooks at 1676015752800 (+2 ms)Region opened successfully at 1676015752800 2023-02-10 07:55:52,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c., pid=150, masterSystemTime=1676015752735 2023-02-10 07:55:52,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:52,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:55:52,801 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:52,801 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:52,801 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:52,802 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1912): 33f54e6bb3ee1980dafaaea7a567c33c/cf1 is initiating minor compaction (all files) 2023-02-10 07:55:52,802 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:52,802 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 33f54e6bb3ee1980dafaaea7a567c33c/cf1 in testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:52,802 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1912): 33f54e6bb3ee1980dafaaea7a567c33c/cf2 is initiating minor compaction (all files) 2023-02-10 07:55:52,802 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 33f54e6bb3ee1980dafaaea7a567c33c/cf2 in testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:52,802 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd-top] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/.tmp, totalSize=34.9 K 2023-02-10 07:55:52,802 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608-top] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/.tmp, totalSize=34.9 K 2023-02-10 07:55:52,802 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015750872 2023-02-10 07:55:52,802 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676015750872 2023-02-10 07:55:52,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:52,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:52,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:52,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c4d2ecf65a7239afbea796e4353ab457, NAME => 'testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.', STARTKEY => '', ENDKEY => '41'} 2023-02-10 07:55:52,803 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015751038 2023-02-10 07:55:52,803 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=33f54e6bb3ee1980dafaaea7a567c33c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:52,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-10 07:55:52,803 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676015751038 2023-02-10 07:55:52,803 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015752803"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015752803"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015752803"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015752803"}]},"ts":"1676015752803"} 2023-02-10 07:55:52,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,803 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015751212 2023-02-10 07:55:52,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-10 07:55:52,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,804 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676015751212 2023-02-10 07:55:52,804 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015751383 2023-02-10 07:55:52,804 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676015751383 2023-02-10 07:55:52,804 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676015751584 2023-02-10 07:55:52,804 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676015751584 2023-02-10 07:55:52,805 INFO [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,805 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676015751749 2023-02-10 07:55:52,805 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676015751749 2023-02-10 07:55:52,805 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015751929 2023-02-10 07:55:52,805 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676015751929 2023-02-10 07:55:52,807 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1 2023-02-10 07:55:52,807 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1 2023-02-10 07:55:52,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-10 07:55:52,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 33f54e6bb3ee1980dafaaea7a567c33c, server=jenkins-hbase4.apache.org,40867,1676015645736 in 217 msec 2023-02-10 07:55:52,807 INFO [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4d2ecf65a7239afbea796e4353ab457 columnFamilyName cf1 2023-02-10 07:55:52,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=33f54e6bb3ee1980dafaaea7a567c33c, ASSIGN in 378 msec 2023-02-10 07:55:52,820 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] throttle.PressureAwareThroughputController(145): 33f54e6bb3ee1980dafaaea7a567c33c#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:52,820 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] throttle.PressureAwareThroughputController(145): 33f54e6bb3ee1980dafaaea7a567c33c#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:52,825 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09-bottom 2023-02-10 07:55:52,827 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a-bottom 2023-02-10 07:55:52,834 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd-bottom 2023-02-10 07:55:52,837 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9-bottom 2023-02-10 07:55:52,838 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/.tmp/cf1/4f1d40dc1afe473ebacfe94b89d2d351 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/4f1d40dc1afe473ebacfe94b89d2d351 2023-02-10 07:55:52,838 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/.tmp/cf2/8fc3c2b9a03741fbb7581556bdd196aa as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/8fc3c2b9a03741fbb7581556bdd196aa 2023-02-10 07:55:52,841 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e-bottom 2023-02-10 07:55:52,843 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 33f54e6bb3ee1980dafaaea7a567c33c/cf1 of 33f54e6bb3ee1980dafaaea7a567c33c into 4f1d40dc1afe473ebacfe94b89d2d351(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:52,843 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 33f54e6bb3ee1980dafaaea7a567c33c/cf2 of 33f54e6bb3ee1980dafaaea7a567c33c into 8fc3c2b9a03741fbb7581556bdd196aa(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:52,843 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 33f54e6bb3ee1980dafaaea7a567c33c: 2023-02-10 07:55:52,843 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 33f54e6bb3ee1980dafaaea7a567c33c: 2023-02-10 07:55:52,843 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c., storeName=33f54e6bb3ee1980dafaaea7a567c33c/cf1, priority=9, startTime=1676015752800; duration=0sec 2023-02-10 07:55:52,844 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c., storeName=33f54e6bb3ee1980dafaaea7a567c33c/cf2, priority=9, startTime=1676015752801; duration=0sec 2023-02-10 07:55:52,844 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:52,844 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:52,844 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524-bottom 2023-02-10 07:55:52,846 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70-bottom 2023-02-10 07:55:52,847 INFO [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(310): Store=c4d2ecf65a7239afbea796e4353ab457/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:52,847 INFO [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,847 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2 2023-02-10 07:55:52,847 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2 2023-02-10 07:55:52,848 INFO [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4d2ecf65a7239afbea796e4353ab457 columnFamilyName cf2 2023-02-10 07:55:52,857 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd-bottom 2023-02-10 07:55:52,859 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608-bottom 2023-02-10 07:55:52,862 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d-bottom 2023-02-10 07:55:52,864 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a-bottom 2023-02-10 07:55:52,867 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f-bottom 2023-02-10 07:55:52,869 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779-bottom 2023-02-10 07:55:52,872 DEBUG [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(539): loaded hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48-bottom 2023-02-10 07:55:52,872 INFO [StoreOpener-c4d2ecf65a7239afbea796e4353ab457-1] regionserver.HStore(310): Store=c4d2ecf65a7239afbea796e4353ab457/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-10 07:55:52,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-10 07:55:52,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:52,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c4d2ecf65a7239afbea796e4353ab457; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65219909, jitterRate=-0.0281476229429245}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-10 07:55:52,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c4d2ecf65a7239afbea796e4353ab457: Running coprocessor pre-open hook at 1676015752804Writing region info on filesystem at 1676015752804Initializing all the Stores at 1676015752804Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015752804Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676015752804Cleaning up temporary data from old regions at 1676015752874 (+70 ms)Cleaning up detritus from prior splits at 1676015752874Running coprocessor post-open hooks at 1676015752876 (+2 ms)Region opened successfully at 1676015752876 2023-02-10 07:55:52,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457., pid=149, masterSystemTime=1676015752735 2023-02-10 07:55:52,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:52,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-10 07:55:52,878 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:52,878 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-10 07:55:52,878 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:52,878 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:52,878 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1912): c4d2ecf65a7239afbea796e4353ab457/cf1 is initiating minor compaction (all files) 2023-02-10 07:55:52,878 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1912): c4d2ecf65a7239afbea796e4353ab457/cf2 is initiating minor compaction (all files) 2023-02-10 07:55:52,878 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c4d2ecf65a7239afbea796e4353ab457/cf1 in testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:52,878 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/.tmp, totalSize=34.9 K 2023-02-10 07:55:52,878 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c4d2ecf65a7239afbea796e4353ab457/cf2 in testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:52,879 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608-bottom] into tmpdir=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/.tmp, totalSize=34.9 K 2023-02-10 07:55:52,879 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015750872 2023-02-10 07:55:52,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:52,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:52,879 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676015750872 2023-02-10 07:55:52,880 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015751038 2023-02-10 07:55:52,880 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676015751038 2023-02-10 07:55:52,880 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c4d2ecf65a7239afbea796e4353ab457, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:52,880 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015751212 2023-02-10 07:55:52,880 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015752880"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676015752880"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676015752880"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676015752880"}]},"ts":"1676015752880"} 2023-02-10 07:55:52,880 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676015751212 2023-02-10 07:55:52,880 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015751383 2023-02-10 07:55:52,881 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676015751383 2023-02-10 07:55:52,881 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015751584 2023-02-10 07:55:52,881 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676015751584 2023-02-10 07:55:52,881 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015751749 2023-02-10 07:55:52,881 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676015751749 2023-02-10 07:55:52,882 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] compactions.Compactor(207): Compacting 5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015751929 2023-02-10 07:55:52,882 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] compactions.Compactor(207): Compacting 7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676015751929 2023-02-10 07:55:52,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-10 07:55:52,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure c4d2ecf65a7239afbea796e4353ab457, server=jenkins-hbase4.apache.org,40867,1676015645736 in 297 msec 2023-02-10 07:55:52,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-10 07:55:52,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c4d2ecf65a7239afbea796e4353ab457, ASSIGN in 454 msec 2023-02-10 07:55:52,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a0f0f141e6bd5bed887272a2c1462d2, daughterA=c4d2ecf65a7239afbea796e4353ab457, daughterB=33f54e6bb3ee1980dafaaea7a567c33c in 950 msec 2023-02-10 07:55:52,891 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] throttle.PressureAwareThroughputController(145): c4d2ecf65a7239afbea796e4353ab457#cf1#compaction#144 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:52,891 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] throttle.PressureAwareThroughputController(145): c4d2ecf65a7239afbea796e4353ab457#cf2#compaction#145 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-10 07:55:52,900 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/.tmp/cf1/ade8cbe8eddf49c1937b108c2f3abdc9 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/ade8cbe8eddf49c1937b108c2f3abdc9 2023-02-10 07:55:52,902 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/.tmp/cf2/c2a5074fcd3c44fa9b9ce5ec10331b05 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c2a5074fcd3c44fa9b9ce5ec10331b05 2023-02-10 07:55:52,903 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c4d2ecf65a7239afbea796e4353ab457/cf1 of c4d2ecf65a7239afbea796e4353ab457 into ade8cbe8eddf49c1937b108c2f3abdc9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:52,904 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c4d2ecf65a7239afbea796e4353ab457: 2023-02-10 07:55:52,904 INFO [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457., storeName=c4d2ecf65a7239afbea796e4353ab457/cf1, priority=9, startTime=1676015752877; duration=0sec 2023-02-10 07:55:52,904 DEBUG [RS:2;jenkins-hbase4:40867-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:52,906 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c4d2ecf65a7239afbea796e4353ab457/cf2 of c4d2ecf65a7239afbea796e4353ab457 into c2a5074fcd3c44fa9b9ce5ec10331b05(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-10 07:55:52,906 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c4d2ecf65a7239afbea796e4353ab457: 2023-02-10 07:55:52,906 INFO [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457., storeName=c4d2ecf65a7239afbea796e4353ab457/cf2, priority=9, startTime=1676015752878; duration=0sec 2023-02-10 07:55:52,906 DEBUG [RS:2;jenkins-hbase4:40867-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-10 07:55:52,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40867] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:47068 deadline: 1676015812936, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2. is not online on jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:53,046 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-10 07:55:53,051 INFO [Listener at localhost/43795] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-10 07:55:53,052 DEBUG [Listener at localhost/43795] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-10 07:55:53,059 WARN [Listener at localhost/43795] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-10 07:55:53,060 INFO [Listener at localhost/43795] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-10 07:55:53,061 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-10 07:55:53,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-10 07:55:53,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-10 07:55:53,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015753064"}]},"ts":"1676015753064"} 2023-02-10 07:55:53,065 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-10 07:55:53,067 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-10 07:55:53,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c4d2ecf65a7239afbea796e4353ab457, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=33f54e6bb3ee1980dafaaea7a567c33c, UNASSIGN}] 2023-02-10 07:55:53,068 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=33f54e6bb3ee1980dafaaea7a567c33c, UNASSIGN 2023-02-10 07:55:53,069 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=33f54e6bb3ee1980dafaaea7a567c33c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:53,069 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015753069"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015753069"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015753069"}]},"ts":"1676015753069"} 2023-02-10 07:55:53,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 33f54e6bb3ee1980dafaaea7a567c33c, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:53,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c4d2ecf65a7239afbea796e4353ab457, UNASSIGN 2023-02-10 07:55:53,073 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c4d2ecf65a7239afbea796e4353ab457, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:53,073 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015753073"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676015753073"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676015753073"}]},"ts":"1676015753073"} 2023-02-10 07:55:53,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure c4d2ecf65a7239afbea796e4353ab457, server=jenkins-hbase4.apache.org,40867,1676015645736}] 2023-02-10 07:55:53,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-10 07:55:53,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:53,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c4d2ecf65a7239afbea796e4353ab457, disabling compactions & flushes 2023-02-10 07:55:53,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:53,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:53,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. after waiting 0 ms 2023-02-10 07:55:53,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:53,223 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd-bottom] to archive 2023-02-10 07:55:53,224 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:53,225 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,226 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,227 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,228 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,229 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,230 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,231 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,232 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a-bottom, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608-bottom] to archive 2023-02-10 07:55:53,233 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:53,234 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,235 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,237 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,238 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,239 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,240 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,241 DEBUG [StoreCloser-testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:55:53,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:53,245 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457. 2023-02-10 07:55:53,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c4d2ecf65a7239afbea796e4353ab457: Waiting for close lock at 1676015753222Running coprocessor pre-close hooks at 1676015753222Disabling compacts and flushes for region at 1676015753222Disabling writes for close at 1676015753222Writing region close event to WAL at 1676015753242 (+20 ms)Running coprocessor post-close hooks at 1676015753244 (+2 ms)Closed at 1676015753245 (+1 ms) 2023-02-10 07:55:53,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:53,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:53,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 33f54e6bb3ee1980dafaaea7a567c33c, disabling compactions & flushes 2023-02-10 07:55:53,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:53,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:53,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. after waiting 0 ms 2023-02-10 07:55:53,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:53,246 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c4d2ecf65a7239afbea796e4353ab457, regionState=CLOSED 2023-02-10 07:55:53,246 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015753246"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015753246"}]},"ts":"1676015753246"} 2023-02-10 07:55:53,247 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd-top] to archive 2023-02-10 07:55:53,248 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:53,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-10 07:55:53,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure c4d2ecf65a7239afbea796e4353ab457, server=jenkins-hbase4.apache.org,40867,1676015645736 in 174 msec 2023-02-10 07:55:53,250 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1ea0116c67d542dc965f226537878d9a.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c4d2ecf65a7239afbea796e4353ab457, UNASSIGN in 182 msec 2023-02-10 07:55:53,251 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9828c324f19d45d983ffec661c06ddf9.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,253 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/a9ba015ce5c6498a94c8ad95e5446524.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,254 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/9f6533583ec34b6fbfb634fedce7083e.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,255 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/1e9553fb84f34311ad19dde00b899c09.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,256 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/d9ce9c92b3894f07811b5749db167d70.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,257 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/5e2085a2592e4c318442ef500c1116cd.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,259 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a-top, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2->hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608-top] to archive 2023-02-10 07:55:53,260 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-10 07:55:53,261 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/9bd55aa75b4b46798b9c502111fc6a8d.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,262 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c81ee2be012143c69382bfd5ad6a3779.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,263 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/cccac44aaa934934bfd5efb2a8408f48.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,264 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,265 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/c64417e9689842f8a0e4bb3e2df8ba3f.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,267 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/aa2f6c2ec0374972b13245df645cd07a.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,268 DEBUG [StoreCloser-testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/7aa5da8155174f658fe3a44a9c44b608.7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-10 07:55:53,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:53,272 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c. 2023-02-10 07:55:53,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 33f54e6bb3ee1980dafaaea7a567c33c: Waiting for close lock at 1676015753246Running coprocessor pre-close hooks at 1676015753246Disabling compacts and flushes for region at 1676015753246Disabling writes for close at 1676015753246Writing region close event to WAL at 1676015753269 (+23 ms)Running coprocessor post-close hooks at 1676015753272 (+3 ms)Closed at 1676015753272 2023-02-10 07:55:53,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:53,274 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=33f54e6bb3ee1980dafaaea7a567c33c, regionState=CLOSED 2023-02-10 07:55:53,274 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676015753273"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676015753273"}]},"ts":"1676015753273"} 2023-02-10 07:55:53,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-10 07:55:53,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 33f54e6bb3ee1980dafaaea7a567c33c, server=jenkins-hbase4.apache.org,40867,1676015645736 in 205 msec 2023-02-10 07:55:53,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-10 07:55:53,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=33f54e6bb3ee1980dafaaea7a567c33c, UNASSIGN in 208 msec 2023-02-10 07:55:53,278 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676015753278"}]},"ts":"1676015753278"} 2023-02-10 07:55:53,279 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-10 07:55:53,281 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-10 07:55:53,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 220 msec 2023-02-10 07:55:53,357 ERROR [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1326607998-172.31.14.131-1676015641262 (Datanode Uuid 929ae40a-10a1-4838-a0c1-b6853964e24b) service to localhost/127.0.0.1:34635 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-10 07:55:53,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-10 07:55:53,366 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-10 07:55:53,366 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-10 07:55:53,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-10 07:55:53,368 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-10 07:55:53,369 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-10 07:55:53,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-10 07:55:53,372 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,372 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:53,372 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:53,374 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/recovered.edits] 2023-02-10 07:55:53,374 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/recovered.edits] 2023-02-10 07:55:53,375 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2, FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/recovered.edits] 2023-02-10 07:55:53,379 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/4f1d40dc1afe473ebacfe94b89d2d351 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf1/4f1d40dc1afe473ebacfe94b89d2d351 2023-02-10 07:55:53,379 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/ade8cbe8eddf49c1937b108c2f3abdc9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf1/ade8cbe8eddf49c1937b108c2f3abdc9 2023-02-10 07:55:53,379 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1e9553fb84f34311ad19dde00b899c09 2023-02-10 07:55:53,381 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/1ea0116c67d542dc965f226537878d9a 2023-02-10 07:55:53,381 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/8fc3c2b9a03741fbb7581556bdd196aa to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/cf2/8fc3c2b9a03741fbb7581556bdd196aa 2023-02-10 07:55:53,381 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c2a5074fcd3c44fa9b9ce5ec10331b05 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/cf2/c2a5074fcd3c44fa9b9ce5ec10331b05 2023-02-10 07:55:53,382 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/5e2085a2592e4c318442ef500c1116cd 2023-02-10 07:55:53,383 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9828c324f19d45d983ffec661c06ddf9 2023-02-10 07:55:53,384 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457/recovered.edits/90.seqid 2023-02-10 07:55:53,384 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/recovered.edits/90.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c/recovered.edits/90.seqid 2023-02-10 07:55:53,384 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/9f6533583ec34b6fbfb634fedce7083e 2023-02-10 07:55:53,385 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/c4d2ecf65a7239afbea796e4353ab457 2023-02-10 07:55:53,385 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/33f54e6bb3ee1980dafaaea7a567c33c 2023-02-10 07:55:53,385 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/a9ba015ce5c6498a94c8ad95e5446524 2023-02-10 07:55:53,386 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf1/d9ce9c92b3894f07811b5749db167d70 2023-02-10 07:55:53,388 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/5df0fd58a1ac41149bf1bbdc2fa146bd 2023-02-10 07:55:53,389 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/7aa5da8155174f658fe3a44a9c44b608 2023-02-10 07:55:53,390 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/9bd55aa75b4b46798b9c502111fc6a8d 2023-02-10 07:55:53,391 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/aa2f6c2ec0374972b13245df645cd07a 2023-02-10 07:55:53,392 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c64417e9689842f8a0e4bb3e2df8ba3f 2023-02-10 07:55:53,393 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/c81ee2be012143c69382bfd5ad6a3779 2023-02-10 07:55:53,394 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48 to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/cf2/cccac44aaa934934bfd5efb2a8408f48 2023-02-10 07:55:53,396 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/recovered.edits/85.seqid to hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/archive/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2/recovered.edits/85.seqid 2023-02-10 07:55:53,396 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/.tmp/data/default/testSplitTableRegion/7a0f0f141e6bd5bed887272a2c1462d2 2023-02-10 07:55:53,397 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-10 07:55:53,398 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-10 07:55:53,400 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-10 07:55:53,401 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-10 07:55:53,402 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-10 07:55:53,402 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-10 07:55:53,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015753402"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:53,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015753402"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:53,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676015753402"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:53,403 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-10 07:55:53,403 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7a0f0f141e6bd5bed887272a2c1462d2, NAME => 'testSplitTableRegion,,1676015750369.7a0f0f141e6bd5bed887272a2c1462d2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c4d2ecf65a7239afbea796e4353ab457, NAME => 'testSplitTableRegion,,1676015751935.c4d2ecf65a7239afbea796e4353ab457.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 33f54e6bb3ee1980dafaaea7a567c33c, NAME => 'testSplitTableRegion,41,1676015751935.33f54e6bb3ee1980dafaaea7a567c33c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-10 07:55:53,404 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-10 07:55:53,404 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676015753404"}]},"ts":"9223372036854775807"} 2023-02-10 07:55:53,406 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-10 07:55:53,407 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-10 07:55:53,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 41 msec 2023-02-10 07:55:53,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37109] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-10 07:55:53,470 INFO [Listener at localhost/43795] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-10 07:55:53,484 INFO [Listener at localhost/43795] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379 (was 377) - Thread LEAK? -, OpenFileDescriptor=836 (was 827) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 104), ProcessCount=168 (was 168), AvailableMemoryMB=3134 (was 3141) 2023-02-10 07:55:53,484 INFO [Listener at localhost/43795] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-10 07:55:53,484 INFO [Listener at localhost/43795] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-10 07:55:53,485 DEBUG [Listener at localhost/43795] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37e887de to 127.0.0.1:60412 2023-02-10 07:55:53,485 DEBUG [Listener at localhost/43795] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:53,486 DEBUG [Listener at localhost/43795] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-10 07:55:53,486 DEBUG [Listener at localhost/43795] util.JVMClusterUtil(257): Found active master hash=1720357835, stopped=false 2023-02-10 07:55:53,487 INFO [Listener at localhost/43795] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:55:53,489 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 07:55:53,489 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 07:55:53,489 INFO [Listener at localhost/43795] procedure2.ProcedureExecutor(629): Stopping 2023-02-10 07:55:53,489 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 07:55:53,489 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:55:53,489 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-10 07:55:53,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:55:53,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:55:53,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:55:53,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-10 07:55:53,489 DEBUG [Listener at localhost/43795] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5baf9fd9 to 127.0.0.1:60412 2023-02-10 07:55:53,490 DEBUG [Listener at localhost/43795] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:53,490 INFO [Listener at localhost/43795] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43281,1676015645662' ***** 2023-02-10 07:55:53,490 INFO [Listener at localhost/43795] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-10 07:55:53,490 INFO [Listener at localhost/43795] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43211,1676015645706' ***** 2023-02-10 07:55:53,490 INFO [Listener at localhost/43795] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-10 07:55:53,490 INFO [Listener at localhost/43795] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,40867,1676015645736' ***** 2023-02-10 07:55:53,490 INFO [Listener at localhost/43795] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-10 07:55:53,490 INFO [RS:0;jenkins-hbase4:43281] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 07:55:53,490 INFO [RS:1;jenkins-hbase4:43211] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 07:55:53,490 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 07:55:53,490 INFO [RS:2;jenkins-hbase4:40867] regionserver.HeapMemoryManager(220): Stopping 2023-02-10 07:55:53,491 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 07:55:53,490 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-10 07:55:53,491 INFO [RS:2;jenkins-hbase4:40867] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 07:55:53,491 INFO [RS:1;jenkins-hbase4:43211] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 07:55:53,491 INFO [RS:0;jenkins-hbase4:43281] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-10 07:55:53,491 INFO [RS:1;jenkins-hbase4:43211] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 07:55:53,491 INFO [RS:2;jenkins-hbase4:40867] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 07:55:53,491 INFO [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:53,491 INFO [RS:0;jenkins-hbase4:43281] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-10 07:55:53,491 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(3304): Received CLOSE for a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:55:53,491 INFO [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:53,491 DEBUG [RS:1;jenkins-hbase4:43211] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7bc645b4 to 127.0.0.1:60412 2023-02-10 07:55:53,491 DEBUG [RS:0;jenkins-hbase4:43281] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2b18cf39 to 127.0.0.1:60412 2023-02-10 07:55:53,492 DEBUG [RS:1;jenkins-hbase4:43211] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:53,492 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:53,492 INFO [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43211,1676015645706; all regions closed. 2023-02-10 07:55:53,492 DEBUG [RS:0;jenkins-hbase4:43281] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:53,492 DEBUG [RS:2;jenkins-hbase4:40867] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x231c9e78 to 127.0.0.1:60412 2023-02-10 07:55:53,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a50233cec7373b84b61f6ef56d6ed02c, disabling compactions & flushes 2023-02-10 07:55:53,492 DEBUG [RS:2;jenkins-hbase4:40867] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:53,492 INFO [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43281,1676015645662; all regions closed. 2023-02-10 07:55:53,492 INFO [RS:2;jenkins-hbase4:40867] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 07:55:53,492 INFO [RS:2;jenkins-hbase4:40867] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 07:55:53,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:55:53,492 INFO [RS:2;jenkins-hbase4:40867] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 07:55:53,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:55:53,492 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-10 07:55:53,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. after waiting 0 ms 2023-02-10 07:55:53,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:55:53,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a50233cec7373b84b61f6ef56d6ed02c 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-10 07:55:53,492 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-10 07:55:53,493 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-10 07:55:53,493 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1479): Online Regions={a50233cec7373b84b61f6ef56d6ed02c=hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c., 1588230740=hbase:meta,,1.1588230740} 2023-02-10 07:55:53,493 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-10 07:55:53,493 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-10 07:55:53,493 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-10 07:55:53,493 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-10 07:55:53,493 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-10 07:55:53,493 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1505): Waiting on 1588230740, a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:55:53,507 DEBUG [RS:1;jenkins-hbase4:43211] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs 2023-02-10 07:55:53,507 INFO [RS:1;jenkins-hbase4:43211] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43211%2C1676015645706:(num 1676015647221) 2023-02-10 07:55:53,507 DEBUG [RS:1;jenkins-hbase4:43211] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:53,507 INFO [RS:1;jenkins-hbase4:43211] regionserver.LeaseManager(133): Closed leases 2023-02-10 07:55:53,508 DEBUG [RS:0;jenkins-hbase4:43281] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs 2023-02-10 07:55:53,508 INFO [RS:0;jenkins-hbase4:43281] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43281%2C1676015645662:(num 1676015647221) 2023-02-10 07:55:53,508 DEBUG [RS:0;jenkins-hbase4:43281] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:53,508 INFO [RS:0;jenkins-hbase4:43281] regionserver.LeaseManager(133): Closed leases 2023-02-10 07:55:53,508 INFO [RS:0;jenkins-hbase4:43281] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-10 07:55:53,508 INFO [RS:1;jenkins-hbase4:43211] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-10 07:55:53,509 INFO [RS:0;jenkins-hbase4:43281] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 07:55:53,509 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 07:55:53,509 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 07:55:53,509 INFO [RS:1;jenkins-hbase4:43211] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-10 07:55:53,509 INFO [RS:0;jenkins-hbase4:43281] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 07:55:53,509 INFO [RS:1;jenkins-hbase4:43211] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-10 07:55:53,509 INFO [RS:0;jenkins-hbase4:43281] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 07:55:53,509 INFO [RS:1;jenkins-hbase4:43211] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-10 07:55:53,511 INFO [RS:1;jenkins-hbase4:43211] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43211 2023-02-10 07:55:53,511 INFO [RS:0;jenkins-hbase4:43281] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43281 2023-02-10 07:55:53,512 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/.tmp/info/df88f941b7964d2a90a450f921ccb6fc 2023-02-10 07:55:53,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 07:55:53,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:53,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:53,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43281,1676015645662 2023-02-10 07:55:53,516 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df88f941b7964d2a90a450f921ccb6fc 2023-02-10 07:55:53,517 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7924b8a6 rejected from java.util.concurrent.ThreadPoolExecutor@7d1b235a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,516 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 07:55:53,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 07:55:53,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 07:55:53,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:53,517 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@55d61fac rejected from java.util.concurrent.ThreadPoolExecutor@7d1b235a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,517 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5b5d542c rejected from java.util.concurrent.ThreadPoolExecutor@5f62cb9f[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:53,517 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43211,1676015645706 2023-02-10 07:55:53,517 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2dce9fac rejected from java.util.concurrent.ThreadPoolExecutor@7d1b235a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,518 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43211,1676015645706] 2023-02-10 07:55:53,518 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43211,1676015645706; numProcessing=1 2023-02-10 07:55:53,520 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43211,1676015645706 already deleted, retry=false 2023-02-10 07:55:53,520 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43211,1676015645706 expired; onlineServers=2 2023-02-10 07:55:53,520 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43281,1676015645662] 2023-02-10 07:55:53,520 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43281,1676015645662; numProcessing=2 2023-02-10 07:55:53,521 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43281,1676015645662 already deleted, retry=false 2023-02-10 07:55:53,521 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43281,1676015645662 expired; onlineServers=1 2023-02-10 07:55:53,526 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/.tmp/rep_barrier/c75f6ab0547c497fa8f92ae2fe4b2eba 2023-02-10 07:55:53,529 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c75f6ab0547c497fa8f92ae2fe4b2eba 2023-02-10 07:55:53,536 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/.tmp/table/a36bb0cbabb141ff9377c33e5e63364b 2023-02-10 07:55:53,539 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a36bb0cbabb141ff9377c33e5e63364b 2023-02-10 07:55:53,539 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/.tmp/info/df88f941b7964d2a90a450f921ccb6fc as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/info/df88f941b7964d2a90a450f921ccb6fc 2023-02-10 07:55:53,542 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df88f941b7964d2a90a450f921ccb6fc 2023-02-10 07:55:53,542 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/info/df88f941b7964d2a90a450f921ccb6fc, entries=59, sequenceid=210, filesize=11.1 K 2023-02-10 07:55:53,543 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/.tmp/rep_barrier/c75f6ab0547c497fa8f92ae2fe4b2eba as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/rep_barrier/c75f6ab0547c497fa8f92ae2fe4b2eba 2023-02-10 07:55:53,545 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c75f6ab0547c497fa8f92ae2fe4b2eba 2023-02-10 07:55:53,545 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/rep_barrier/c75f6ab0547c497fa8f92ae2fe4b2eba, entries=24, sequenceid=210, filesize=7.5 K 2023-02-10 07:55:53,546 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/.tmp/table/a36bb0cbabb141ff9377c33e5e63364b as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/table/a36bb0cbabb141ff9377c33e5e63364b 2023-02-10 07:55:53,548 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a36bb0cbabb141ff9377c33e5e63364b 2023-02-10 07:55:53,548 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/table/a36bb0cbabb141ff9377c33e5e63364b, entries=36, sequenceid=210, filesize=8.0 K 2023-02-10 07:55:53,549 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 56ms, sequenceid=210, compaction requested=false 2023-02-10 07:55:53,550 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 07:55:53,550 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 07:55:53,550 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-10 07:55:53,553 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-10 07:55:53,553 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:53,553 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-10 07:55:53,554 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-10 07:55:53,554 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676015753493Running coprocessor pre-close hooks at 1676015753493Disabling compacts and flushes for region at 1676015753493Disabling writes for close at 1676015753493Obtaining lock to block concurrent updates at 1676015753493Preparing flush snapshotting stores in 1588230740 at 1676015753493Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676015753493Flushing stores of hbase:meta,,1.1588230740 at 1676015753494 (+1 ms)Flushing 1588230740/info: creating writer at 1676015753494Flushing 1588230740/info: appending metadata at 1676015753501 (+7 ms)Flushing 1588230740/info: closing flushed file at 1676015753502 (+1 ms)Flushing 1588230740/rep_barrier: creating writer at 1676015753517 (+15 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676015753520 (+3 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676015753520Flushing 1588230740/table: creating writer at 1676015753529 (+9 ms)Flushing 1588230740/table: appending metadata at 1676015753531 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676015753531Flushing 1588230740/info: reopening flushed file at 1676015753540 (+9 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676015753543 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1676015753546 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 56ms, sequenceid=210, compaction requested=false at 1676015753549 (+3 ms)Writing region close event to WAL at 1676015753551 (+2 ms)Running coprocessor post-close hooks at 1676015753553 (+2 ms)Closed at 1676015753554 (+1 ms) 2023-02-10 07:55:53,554 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-10 07:55:53,618 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:53,618 INFO [RS:0;jenkins-hbase4:43281] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43281,1676015645662; zookeeper connection closed. 2023-02-10 07:55:53,618 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3d781207 rejected from java.util.concurrent.ThreadPoolExecutor@7d1b235a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,618 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43281-0x1003d3915cd0001, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:53,618 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@36caf29b rejected from java.util.concurrent.ThreadPoolExecutor@7d1b235a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,618 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@40b1021c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@40b1021c 2023-02-10 07:55:53,694 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1505): Waiting on a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:55:53,718 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:53,718 INFO [RS:1;jenkins-hbase4:43211] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43211,1676015645706; zookeeper connection closed. 2023-02-10 07:55:53,718 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a7d086c rejected from java.util.concurrent.ThreadPoolExecutor@5f62cb9f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,718 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:43211-0x1003d3915cd0002, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:53,718 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6b182ca8] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6b182ca8 2023-02-10 07:55:53,719 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6247cbe1 rejected from java.util.concurrent.ThreadPoolExecutor@5f62cb9f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:53,894 DEBUG [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1505): Waiting on a50233cec7373b84b61f6ef56d6ed02c 2023-02-10 07:55:53,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/.tmp/info/ce21494b6058422f90c8ab0a9e7b6ec6 2023-02-10 07:55:53,911 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/.tmp/info/ce21494b6058422f90c8ab0a9e7b6ec6 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/info/ce21494b6058422f90c8ab0a9e7b6ec6 2023-02-10 07:55:53,914 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/info/ce21494b6058422f90c8ab0a9e7b6ec6, entries=2, sequenceid=6, filesize=4.8 K 2023-02-10 07:55:53,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a50233cec7373b84b61f6ef56d6ed02c in 423ms, sequenceid=6, compaction requested=false 2023-02-10 07:55:53,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/data/hbase/namespace/a50233cec7373b84b61f6ef56d6ed02c/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-10 07:55:53,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:53,920 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:55:53,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a50233cec7373b84b61f6ef56d6ed02c: Waiting for close lock at 1676015753492Running coprocessor pre-close hooks at 1676015753492Disabling compacts and flushes for region at 1676015753492Disabling writes for close at 1676015753492Obtaining lock to block concurrent updates at 1676015753492Preparing flush snapshotting stores in a50233cec7373b84b61f6ef56d6ed02c at 1676015753492Finished memstore snapshotting hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676015753493 (+1 ms)Flushing stores of hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. at 1676015753493Flushing a50233cec7373b84b61f6ef56d6ed02c/info: creating writer at 1676015753493Flushing a50233cec7373b84b61f6ef56d6ed02c/info: appending metadata at 1676015753497 (+4 ms)Flushing a50233cec7373b84b61f6ef56d6ed02c/info: closing flushed file at 1676015753497Flushing a50233cec7373b84b61f6ef56d6ed02c/info: reopening flushed file at 1676015753912 (+415 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for a50233cec7373b84b61f6ef56d6ed02c in 423ms, sequenceid=6, compaction requested=false at 1676015753915 (+3 ms)Writing region close event to WAL at 1676015753916 (+1 ms)Running coprocessor post-close hooks at 1676015753919 (+3 ms)Closed at 1676015753920 (+1 ms) 2023-02-10 07:55:53,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676015647681.a50233cec7373b84b61f6ef56d6ed02c. 2023-02-10 07:55:54,065 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-10 07:55:54,065 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-10 07:55:54,094 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,40867,1676015645736; all regions closed. 2023-02-10 07:55:54,099 DEBUG [RS:2;jenkins-hbase4:40867] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs 2023-02-10 07:55:54,099 INFO [RS:2;jenkins-hbase4:40867] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40867%2C1676015645736.meta:.meta(num 1676015647398) 2023-02-10 07:55:54,103 DEBUG [RS:2;jenkins-hbase4:40867] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/oldWALs 2023-02-10 07:55:54,103 INFO [RS:2;jenkins-hbase4:40867] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C40867%2C1676015645736:(num 1676015647220) 2023-02-10 07:55:54,103 DEBUG [RS:2;jenkins-hbase4:40867] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:54,103 INFO [RS:2;jenkins-hbase4:40867] regionserver.LeaseManager(133): Closed leases 2023-02-10 07:55:54,103 INFO [RS:2;jenkins-hbase4:40867] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-10 07:55:54,103 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 07:55:54,104 INFO [RS:2;jenkins-hbase4:40867] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40867 2023-02-10 07:55:54,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-10 07:55:54,106 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,40867,1676015645736 2023-02-10 07:55:54,106 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3b2ca41c rejected from java.util.concurrent.ThreadPoolExecutor@3f1327a1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:54,108 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,40867,1676015645736] 2023-02-10 07:55:54,108 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,40867,1676015645736; numProcessing=3 2023-02-10 07:55:54,109 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,40867,1676015645736 already deleted, retry=false 2023-02-10 07:55:54,109 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,40867,1676015645736 expired; onlineServers=0 2023-02-10 07:55:54,109 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37109,1676015644499' ***** 2023-02-10 07:55:54,109 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-10 07:55:54,110 DEBUG [M:0;jenkins-hbase4:37109] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27b799f0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-10 07:55:54,110 INFO [M:0;jenkins-hbase4:37109] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37109,1676015644499 2023-02-10 07:55:54,110 INFO [M:0;jenkins-hbase4:37109] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37109,1676015644499; all regions closed. 2023-02-10 07:55:54,110 DEBUG [M:0;jenkins-hbase4:37109] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-10 07:55:54,110 DEBUG [M:0;jenkins-hbase4:37109] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-10 07:55:54,110 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-10 07:55:54,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676015646844] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676015646844,5,FailOnTimeoutGroup] 2023-02-10 07:55:54,110 DEBUG [M:0;jenkins-hbase4:37109] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-10 07:55:54,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676015646845] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676015646845,5,FailOnTimeoutGroup] 2023-02-10 07:55:54,110 INFO [M:0;jenkins-hbase4:37109] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-10 07:55:54,110 INFO [M:0;jenkins-hbase4:37109] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-10 07:55:54,110 INFO [M:0;jenkins-hbase4:37109] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-10 07:55:54,110 DEBUG [M:0;jenkins-hbase4:37109] master.HMaster(1502): Stopping service threads 2023-02-10 07:55:54,110 INFO [M:0;jenkins-hbase4:37109] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-10 07:55:54,111 ERROR [M:0;jenkins-hbase4:37109] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] 2023-02-10 07:55:54,111 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-10 07:55:54,111 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-10 07:55:54,111 INFO [M:0;jenkins-hbase4:37109] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-10 07:55:54,111 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-10 07:55:54,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-10 07:55:54,111 DEBUG [M:0;jenkins-hbase4:37109] zookeeper.ZKUtil(398): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-10 07:55:54,111 WARN [M:0;jenkins-hbase4:37109] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-10 07:55:54,111 INFO [M:0;jenkins-hbase4:37109] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-10 07:55:54,111 INFO [M:0;jenkins-hbase4:37109] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-10 07:55:54,111 DEBUG [M:0;jenkins-hbase4:37109] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-10 07:55:54,111 INFO [M:0;jenkins-hbase4:37109] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:55:54,111 DEBUG [M:0;jenkins-hbase4:37109] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:55:54,111 DEBUG [M:0;jenkins-hbase4:37109] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-10 07:55:54,112 DEBUG [M:0;jenkins-hbase4:37109] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:55:54,112 INFO [M:0;jenkins-hbase4:37109] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.91 KB heapSize=718.99 KB 2023-02-10 07:55:54,139 INFO [M:0;jenkins-hbase4:37109] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.91 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1a1be3a8c16b4231be11b2fcd2f7f517 2023-02-10 07:55:54,142 INFO [M:0;jenkins-hbase4:37109] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a1be3a8c16b4231be11b2fcd2f7f517 2023-02-10 07:55:54,143 DEBUG [M:0;jenkins-hbase4:37109] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1a1be3a8c16b4231be11b2fcd2f7f517 as hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1a1be3a8c16b4231be11b2fcd2f7f517 2023-02-10 07:55:54,147 INFO [M:0;jenkins-hbase4:37109] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a1be3a8c16b4231be11b2fcd2f7f517 2023-02-10 07:55:54,147 INFO [M:0;jenkins-hbase4:37109] regionserver.HStore(1080): Added hdfs://localhost:34635/user/jenkins/test-data/be2ced0a-3d77-7008-b606-c812b69d09a1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1a1be3a8c16b4231be11b2fcd2f7f517, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-10 07:55:54,148 INFO [M:0;jenkins-hbase4:37109] regionserver.HRegion(2947): Finished flush of dataSize ~607.91 KB/622496, heapSize ~718.98 KB/736232, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 36ms, sequenceid=1282, compaction requested=false 2023-02-10 07:55:54,149 INFO [M:0;jenkins-hbase4:37109] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-10 07:55:54,149 DEBUG [M:0;jenkins-hbase4:37109] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676015754111Disabling compacts and flushes for region at 1676015754111Disabling writes for close at 1676015754112 (+1 ms)Obtaining lock to block concurrent updates at 1676015754112Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676015754112Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622496, getHeapSize=736232, getOffHeapSize=0, getCellsCount=1499 at 1676015754112Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676015754112Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676015754112Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676015754115 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676015754115Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676015754143 (+28 ms)Finished flush of dataSize ~607.91 KB/622496, heapSize ~718.98 KB/736232, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 36ms, sequenceid=1282, compaction requested=false at 1676015754148 (+5 ms)Writing region close event to WAL at 1676015754149 (+1 ms)Closed at 1676015754149 2023-02-10 07:55:54,157 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-10 07:55:54,157 INFO [M:0;jenkins-hbase4:37109] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-10 07:55:54,157 INFO [M:0;jenkins-hbase4:37109] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37109 2023-02-10 07:55:54,158 DEBUG [M:0;jenkins-hbase4:37109] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37109,1676015644499 already deleted, retry=false 2023-02-10 07:55:54,208 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:54,208 INFO [RS:2;jenkins-hbase4:40867] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,40867,1676015645736; zookeeper connection closed. 2023-02-10 07:55:54,208 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1573e870 rejected from java.util.concurrent.ThreadPoolExecutor@3f1327a1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:54,209 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): regionserver:40867-0x1003d3915cd0003, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:54,209 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6c9ec34e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6c9ec34e 2023-02-10 07:55:54,209 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6da9aa6c rejected from java.util.concurrent.ThreadPoolExecutor@3f1327a1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:54,209 INFO [Listener at localhost/43795] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-10 07:55:54,308 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:54,308 INFO [M:0;jenkins-hbase4:37109] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37109,1676015644499; zookeeper connection closed. 2023-02-10 07:55:54,308 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4a370004 rejected from java.util.concurrent.ThreadPoolExecutor@50b483f3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:54,309 DEBUG [Listener at localhost/43795-EventThread] zookeeper.ZKWatcher(600): master:37109-0x1003d3915cd0000, quorum=127.0.0.1:60412, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-10 07:55:54,309 ERROR [Listener at localhost/43795-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4d2a8447 rejected from java.util.concurrent.ThreadPoolExecutor@50b483f3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-10 07:55:54,310 WARN [Listener at localhost/43795] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 07:55:54,479 INFO [Listener at localhost/43795] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 07:55:54,507 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-10 07:55:54,582 WARN [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 07:55:54,582 WARN [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1326607998-172.31.14.131-1676015641262 (Datanode Uuid e71256cb-90c7-4766-bfdc-18ad52545fac) service to localhost/127.0.0.1:34635 2023-02-10 07:55:54,585 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data5/current/BP-1326607998-172.31.14.131-1676015641262] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 07:55:54,586 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data6/current/BP-1326607998-172.31.14.131-1676015641262] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 07:55:54,588 WARN [Listener at localhost/43795] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 07:55:54,592 INFO [Listener at localhost/43795] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 07:55:54,601 WARN [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-10 07:55:54,601 WARN [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1326607998-172.31.14.131-1676015641262 (Datanode Uuid 1d48e5e3-95b6-462f-8431-b0ebec0b2069) service to localhost/127.0.0.1:34635 2023-02-10 07:55:54,602 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data3/current/BP-1326607998-172.31.14.131-1676015641262] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 07:55:54,603 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data4/current/BP-1326607998-172.31.14.131-1676015641262] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 07:55:54,603 WARN [Listener at localhost/43795] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-10 07:55:54,609 INFO [Listener at localhost/43795] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 07:55:54,676 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-10 07:55:54,676 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-10 07:55:54,712 WARN [BP-1326607998-172.31.14.131-1676015641262 heartbeating to localhost/127.0.0.1:34635] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1326607998-172.31.14.131-1676015641262 (Datanode Uuid 929ae40a-10a1-4838-a0c1-b6853964e24b) service to localhost/127.0.0.1:34635 2023-02-10 07:55:54,713 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data1/current/BP-1326607998-172.31.14.131-1676015641262] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 07:55:54,713 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f149bb8c-282a-a08f-44d6-023a83b9277d/cluster_652903ca-0399-3703-4dbc-9280186c2063/dfs/data/data2/current/BP-1326607998-172.31.14.131-1676015641262] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-10 07:55:54,733 INFO [Listener at localhost/43795] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-10 07:55:54,919 INFO [Listener at localhost/43795] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-10 07:55:55,166 INFO [Listener at localhost/43795] hbase.HBaseTestingUtility(1293): Minicluster is down